Interview of Danny Faught

This interview is conducted by WhatIsTesting forum (http://www.whatistesting.com/forum/phpBB3/viewtopic.php?f=35&t=2)

Danny Faught is widely known in testing circles because of the various lists that he maintains, including the Test Tools List, Testing Contractors and Consultants List, and the Testing Courses List on http://www.testingfaqs.org.

He is owner of Tejas Software consulting http://www.tejasconsulting.com. In this interview he shares his ideas, passion and advice with the testers.

If you want to ask him more questions or you want to send any feedback on this interview, please reply to this post.

Q1. Danny, please tell us something about yourself, your background, your interests and things that you like to do in your free time. Please also tell something about Tejas Software Consulting.

First let me say that I've enjoyed reading the interviews on whatistesting.com and I'm honored to have the opportunity to share my thoughts with you.

I started as a software tester right after I got my bachelor's degree. It was really happenstance - I learned nothing whatsoever about testing in college. I was offered a job doing testing for supercomputer operating systems. This was by far the most interesting product I had the chance to work with at the time, so I thought I would try being a tester. And besides getting sidetracked doing process improvement for a while, I've been a tester ever since.

I'll never know if I would have gravitated toward testing if I had started out as a programmer instead. In a way, though, I did start out as a programmer because I had a co-op job as a programmer while I was in college.

I went into business in early 2001, the day after I was laid off by a consulting firm I was working for. Since then I've done a variety of things, including training, consulting, editing, publishing, and freelance writing. The business is a one-man show and I don't plan to hire any additional employees. You can learn more than you ever wanted to know about Tejas Software Consulting from my yearly wrap-up articles in the February/March issues of my newsletter (see http://tejasconsulting.com/newsletter/) or from the About Tejas Software Consulting web page (http://tejasconsulting.com/about/).

I have a personal interest that's a pretty good balance to the esoteric world I work in. I'm a bit of an "eco-freak," as those who've carefully read my newsletter might have figured out. I'm an avid composter, actually, I'm a Certified Texas Master Composter. My yard and my garden are organic. And I drive a hybrid electric car, a Toyota Prius. I could go on. This passion comes through in my work, where, for example, I always use recycled paper, and I've started asking others to do the same (so far unsuccessfully).

Another personal interest is homeschooling. My wife and I have been homeschooling our oldest daughter for almost a year, and we're loving it. I've started writing articles about the subject, and I teach a programming class for homeschool kids.

Q2. You maintain various lists such as on Test Tools, Testing Contractors and Consultants List etc. at http://testingfaqs.org. What drives you to maintain these lists?

Good question! I've maintained a variety of FAQs ever since I created the comp.software.testing FAQ. Usually I get started with this sort of project because I have a need to reference the information myself. For example, I also maintain a list of software-related organizations and other resources that are local to the North Texas area (http://tejasconsulting.com/resources/ntex.html). I took over the maintenance of the FAQs that are now on testingfaqs.org when Brian Marick said that he didn't want to do it any more. They had such a history, and were such an important part of the testing community, I couldn't stand to see them fall by the wayside.

I really enjoy the research required put these FAQs together, especially the Boneyard page, where I track historical information about companies, products, and services.

It also helps that I occasionally hear from someone who tells me how useful testingfaqs.org has been to them. That sort of feedback really makes my day. I'm embarrassed that I'm woefully behind on posting new entries that people have sent me. It's very important to me that I review the entries, make sure they're in the right category, and remove any superfluous hype. That takes a tremendous amount of time.

Q3. Do you think you have been getting enough support from people in this endeavor? What kind of support would you like to see for these lists from people?

I could certainly keep the site more up to date if I had help. I have had a few people help with things like the CGI scripting, and I've been starting to get a slow trickle of people who are pointing out new freeware tools. I think I could get more help if I could make it easier for people to help. I've thought about using a wiki, though it would be hard to maintain a consistent format on a wiki.

I'm very interested in hearing from people who would like to help, and we can be creative with the way they contribute. Of course, people can also contribute financially by buying a text ad or by sending a donation.

Q4.What is your opinion on the state of software testing today and where do you see it going?

I call myself a "Software Alchemist" because I think we're still in the dark ages of software. I believe it will be many decades before software development becomes a true engineering discipline like it should be. I'm not trying to be self-righteous in this pronouncement; I don't know all the answers either.

The average software tester is still fairly disconnected from the wealth of resources that are available. Most haven't been to a training course or university course that focuses on testing. Many haven't even seen a book about testing - they're still very hard to find in the brick and mortar bookstores. Test managers are in a similar position.

So testing often isn't much better than what common sense tells people to do. While companies are usually surviving in this mode, they could be much more efficient and produce much better products if they tapped into the accumulated knowledge that's out there.

It helps that there's a growing library of free information about testing on the web, but sorting through it to find what's relevant and what's worthwhile advice is difficult, especially for a beginner.

Where is it going? Looks like it's going to India. I'm getting an increasing number of queries from people in India, asking how to get started in testing. (One factor for this may be that my company name happens to have an interesting meaning in several Indian languages.) They seem to have an even more difficult time than average in getting access to resources.

I don't have a business background, so I won't try to guess what's going to happen to the industry.

Q5. What are the things a person who wants to choose testing as a career should do to prepare himself for it?

Do you think this question needs to be added to the testing FAQ?
I've had so many people ask this question that I've been setting their emails aside, planning to write an article on this topic to answer them all at once.

I'm going to write that article soon, but unfortunately a goblin deleted those emails, so I hope those who asked have joined my newsletter mailing list so they'll see the answer. I'm also working on a course that introduces people to testing as a career. The hard part is finding any reason to give them hope of finding an entry-level job.

Q6. What things should a novice tester do to enhance his/her testing skills for a better growth in the career?

Learn a general-purpose programming language if you don't know one already, especially a scripting language like Perl, Python, or Ruby, and also the language that the developers on your project are using. Start doing homebrew automation, automating small everyday tasks, even if test execution itself is still manual. There are things you can automate with only a few minutes or hours of effort, so you don't have to get anyone's approval to do it. People will start to ask, "Wow, how did you do that?"

Read books and articles about testing, network with people who do what you do, and attend conferences and workshops. Also learn consulting skills, even if you're not a consultant (see "The Secrets of Consulting" by Jerry Weinberg). It's easy to learn some nifty new way to do testing. It's much harder to judge whether the technique is really appropriate for your organization, and then to inject a new technique into the organization's culture.

Adopt a mentor, even if your company doesn't have a formal mentoring program. This might be somebody in your company, or someone on the other side of the world whom you've formed a relationship with.

Q7. Where do you see testing tools technology heading? Do you foresee some open-source testing tools which can give COTS tools a run for their money? How do you see your reviews of open testware tools helping the testing community?

The simpler commercial tools are probably feeling the heat from the open source community. For example, you don't see many companies trying to sell a standalone html link checker now, and I believe the availability of free link checking tools has had an influence. The commercial tools usually have several additional features added in order to make them marketable.

At the high end, the more complex types of tools probably aren't causing any commercial vendors to lose sleep yet. The commercial tools tend to have more features and are at least somewhat more reliable. I've looked at a few free load testing tools, for example. A number of expert users are utilizing these tools with good results and saving their companies tens of thousands of dollars in licensing fees. However, the tools are cranky and tend to be difficult to learn, so they're not ready for mainstream usage. Because people who do load testing have to be very experienced testers anyway, I think this is going to be the first type of complex test tool to get a foot hold. They're not just quite there yet.

Then there's homebrew automation, much of which is well below the radar of the commercial market. It's often easier to write a few lines of code to solve a specific automation problem than to find a relevant open source tool and learn how to use it. But open source also plays a big role here. The commercial vendors who have good external hooks in their tools and that don't use proprietary "vendorscript" languages are best-suited to integrate with a test team that's doing a lot of homebrew. But these teams may also be avoiding commercial tools entirely.

What I'm doing with Open Testware Reviews is finding the most promising open source and freeware tools, making people aware of them, and putting them through a grueling review. Rather than ride the wave of open source hype, I want to see what these tools can actually do and learn how difficult it will be for the average toolsmith to use them. I'm also hoping that as I share my findings with the project teams that maintain the tools, my work will help them improve the tools and make them more suitable for mainstream usage.

Q8. What is your advice to people who want to learn automated testing tools on their own? How can they do it without spending a lot of money?

That's a tough one. So many job postings demand years of experience with the big-dog tools. It's very difficult to get access to these tools without spending a lot of money on licenses. I've even seen a firm that joined a tool vendor's partner program and still couldn't get sufficient access to the tools for training purposes for its employees.

The obvious answer is to learn open source tools instead. Even better, use the tools to test another open source program, so you're doing real work rather than just playing. At least for me, I learn better when I have a real-world problem to solve. I use open source tools when I teach courses about test automation, because the participants can go back to their office and immediately start using the tool.

Unfortunately, many employers don't understand how experience with one example of a particular type of tool might enable you to easily learn how to do the same type of testing with a different tool. Maybe when an open source tool becomes as popular as its commercial counterparts, or if we get back to a situation where there are more testing jobs than candidates, open source tool experience will be a valuable addition to a resume. Certainly, there are some niche consulting firms that are having a go at marketing their skills with certain open source test tools.

Q9. You maintain a stress testing tool(stress-driver) on http://sourceforge.net/projects/stress-driver/. Can you please tell us something about this tool, how it can be useful and under what circumstances?

I originally designed this tool to drive stress tests for an operating system when I worked for Convex Computer Corporation. My team ended up using it for a wide variety of stress tests. After it was released under an open source license, I ported it to Linux and Windows and created a project for it on SourceForge. About 200 people have downloaded it so far.

The stress_driver is designed to run multiple invocations of a single test program. There are few restrictions on what the test program itself can do. I've used it to test filesystems, process management, threads, and I it used as one layer of a complex reliability test that simulated interactive user logins. You specify how many copies of the test programs to run, or a range to vary it within. You can let the individual tests run as long as they want to, or the stress_driver can stop them after a randomly determined time interval. There are many other features designed to make stress_driver flexible.

I can't guarantee that the tool will be useful for anything other than operating system testing since I haven't expanded its use beyond that context. But I think it's well-suited for a variety of situations where you have a single program that you want to run multiple copies of, with randomized variations. The test program can of course launch as complex a scenario as you want.

I haven't yet heard from anyone who has downloaded the tool, but the infrastructure is there on SourceForge if anyone wants to discuss any questions they have. Like any tool author, I'd love to hear how people have tried to use it.

Q10. What are top 5 testing books that you would recommend to people?

"Software Testing" by Ron Patton. This is a very well-written and gentle introduction to the subject. Those who already have a good grasp of testing can skip it.
"Lessons Learned in Software Testing" by Cem Kaner, James Bach, and Bret Pettichord. I've recently started recommending this one above the perennial favorite below. Very practical, up-to-date, and useful guidance.
"Testing Computer Software, 2nd edition" by Cem Kaner, Jack Falk, and Hung Nguyen. Generally recognized as the best and best-selling testing book.
"The Secrets of Consulting" by Jerry Weinberg. Testers need to be able to influence people in their organization the same way consultants do.
"Test-Driven Development" by Kent Beck. This is a growing trend among developers that stands to fundamentally affect the way software is developed and what testers' roles are.

Q11. If you were asked to name five persons who have made maximum contribution to testing, what names would you suggest?

This is actually a timely question. I'm writing a series of testing resource guides, and some of the early feedback I got was that I should add a "Who's Who" of testing. This is always a dangerous thing to do, because of the notable people who are invariably left off the list, but I'm going to give it a try.

It would take a lot of research for me to say with any confidence who has had the greatest impact in forming the collective wisdom about testing that has accumulated since the dawn of software. I'll just tell you five of the people who have had the biggest impact on me.

Boris Beizer is the grandfather of software testing. He was one of my first mentors. Many people find him difficult to like, but he nonetheless has had a big impact on the field.
Cem Kaner has done many things, from writing practical and influential books, to volunteering for the cause of software consumer protection, and now he's teaching a new generation of testers at Florida Tech.
Edward Miller facilitated my first face-to-face meetings with dozens of fascinating people in this field. His Quality Week conferences opened my eyes to many new ideas, and taught me the importance of meeting people.
Brian Marick has introduced refreshing new ideas, and has told more than one emperor that he has no clothes. I've often tried to emulate his pragmatic approach to evaluating testing techniques.
Many people still haven't heard of Jerry Weinberg. He prefers to have a profound impact on a few people rather than spreading his influence as thinly and as broadly as possible. He's certainly had a profound effect on me. But his books deserve more attention than they're getting. While he sometimes talks about software testing and practically everything he says can have some bearing on quality, what he really contributes is teaching us that all the issues are really people issues, not technology problems.
There are so many other people who have more recently affected my work and have contributed to the field as a whole. I started to list them, but the list just wouldn't stop growing, so I think I'd be safer to stop here for now. You'll have to wait for the Who's Who to see the rest.

If you were your own interviewer what questions would you like to ask yourself? They are:

1. What have you been up to lately?
I've been doing a lot of writing lately, and I dearly love doing it. But I've also finally gotten serious about convincing people to help me with writing Open Testware Reviews. So to some extent I'll take on the role of editor. I'm excited to be able to represent some different voices in Open Testware Reviews.

I've started to try to put together my first book. I really need to get a book published in order to look like a credible consultant. I have several ideas on the short list; I decided the first one to tackle is to basically take the FAQs I've been involved with over the years and put them in book form. It's frustrating seeing that pretty much all of the testing FAQs that are out there on the web now are incomplete and outdated. I can do much better than that. But I'm realizing that keeping this information up-to-date might be even harder in book form.

2. So how will you publish these lists of data that need to be updated?

I've put a lot thought into that. What I'd like to do is to publish a series of industry reports that are updated yearly. People who buy the reports will get updates for a year, and perhaps a discount on the next year's edition. There does need to be a dynamic element to it. I'm tentatively planning for the first one to include lists of conferences, professional organizations, online forums, and that Who's Who I mentioned earlier.

I'm including value-added information so that the reports aren't just lists of data. This will include cross-references, annotations, and commentary on how best to utilize the resources. The commentary may eventually grow into a book, but for now it looks like the book idea will have to wait.

Right now I'm building an international review panel that can help me make sure that the testing industry all across the world is well-represented in the reports.

3. What message do you want to leave testers with?

Testing is fun! We get to make a mess of the software and then someone else cleans up the mess. We get paid to do devious things on purpose. So have fun with it - now get back to work and break something.