Software development for non-technical founders: Testing early and all the time
So far in this series, we discussed the complexities involved in software development, the common pitfalls to avoid, and a blueprint for navigating the software development process for the non-technical founders. Then, we discussed the importance of getting the requirement right in interactive software development.
In this article, we discuss the different types of testing and their importance in developing a product that your customer needs.
Testing overview
In your product development journey, you need to test different aspects of the product, such as ideas, value proposition, prototypes, and the actual software product itself. Testing each requires different tools and methodologies.
Figure 01: Different types of testing involved when developing your product
Explaining testing ideas and value proposition is out of scope for our article. However, you can gain knowledge in testing them using these two books we have provided below:
Five Discovery Skills that Distinguish Great Innovators
The later book also discusses testing the prototypes.
Testing prototypes
Many consider the prototype as an MVP. However, the prototype is nothing but a mock test of your ideas in an early stage of the product, and it’s not done one time but many times for each new feature you’d like to introduce to your potential customers. It allows you to collect valuable feedback from your potential customers even before your product is developed — saving you time and money.
In this section, we discuss four types of prototypes appropriate for software development (however, you are free to research and pursue other types of prototypes as well).
Horizontal — shows the design from your customer’s perspective to the engineers to help them understand the human interfaces and interactions. Horizontal prototypes generally show menus, inputs and outputs, and other components that allow users to interact.
Wireframe — is our favorite kind of prototyping. It is the skeleton of your product, developed rapidly — sometimes even in front of your customers to test product ideas and gain useful feedback. It can range from using pencil and paper on a napkin to using software that helps you create and destroy screens and workflows at will.
Mock-up — is usually created after wireframe-based testing to present the visual idea of your product. Sometimes a mock-up can be put together using different products such as WordPress with plugins. The focus is generally on the visual aesthetics of your product, but it can also be an unpolished early version of your product to test your ideas.
Feasibility prototype — is usually done after testing your wireframes and possibly mock-up to determine the technical feasibility of your product, such as performance, compatibility of components, tools, and others. This testing is performed with your technical lead (the senior-most member of your software development team).
We strongly recommend that you do a minimum of wireframe testing with your potential customers of all the above-listed prototype testing. As a startup founder, you have many other things to do, and so conducting all this testing will not be feasible at the early stage. However, as discussed earlier, you need to continue developing and testing your prototypes even after releasing your product to the market. You should consider adding more types of prototypes to make sure you build on the early traction you gained with your customers in such a situation.
Here are some of the software products that will help you create wireframes and test them with your potential customers:
Wireframesketcher — this is a desktop product with features that allow you to share screens with your customers online for testing. This is our favorite software, although there are more cloud-based convenient ones.
Balsamiq wireframe — this is a cloud-based software for wireframing
Figma is another cloud-based wireframe product that also helps you develop mock-ups
Adobe XD is another cloud-based software for wireframing and mock-ups
Proto.io is another cloud-based software for wireframing and mock-ups. This also works with user testing platforms to roll out your wireframes to thousands of users for testing and feedback on the user experience.
You need to consider a few important things when choosing wireframing and mock-up software — the number of screens you will have to design. Most of the cloud-based software we have tested cannot handle more than 100 screens when you are in design mode. One of the product wireframe prototypes we developed consists of nearly 750 screens. It couldn’t survive the cloud-based products, so we had to switch to desktop (or laptop) based wireframing software such as Wireframesketcher.
Another consideration is whether to combine wireframes and mock-ups in a single prototype. We advise against combining both during the first few iterations with your potential customers since you do not want to distract them from focusing on the workflows and the user experience. However, once you are past the initial stage, mock-ups are great to test the look and feel of your product.
Software product
Software product testing demands multiple articles on its own. However, we summarized it for you in this article.
Depending on where you look, the types of testing you need to do on your software range from six to 17. However, we focus on six types of testing that are important for ensuring your product does not cause additional problems to your early adopters that they desert your product.
Figure 03: Different tests performed in a software development
Unit tests — these are the basic testing performed continuously during software development. They test the individual piece of code written to ensure that the later changes do not break the earlier codes. This is one of the most important testings you need to perform in your software development by your teams. The code to automate unit testing is written by your software developers (or testers — if you have enough money to hire dedicated software testers).
User interface tests — the use of software applications, especially those hosted over the internet or developed for mobile devices, are complex. Hence, user interfaces tend to break when used in different devices due to their compatibility or something unknown at the time of development. Hence, testing the core functionalities (at the bare minimum) in different browsers and mobile devices is important before releasing them to your customers or early adopters.
Here you usually test the menu placements, aesthetics of your product, responsive design, buttons, and other components that your customers see and interact with. If your site relies heavily on javascript to deliver its user interface (or even business functionality), this test will reveal how they respond in a restrictive browser. Also, knowing your customer segment will help you know the breadth and depth of your UI testing. If your customers are primarily iOS users, then spending more time testing iOS UI makes more sense.
Integration testing — is for testing the workflows between different modules and different components of your product. For example, when your customer logs into your software product, they need to receive a security token which is performed by a background process in your product. Suppose the background process does not work based on your customer showing an intend to log in. In that case, your customers will be waiting to log in indefinitely and eventually stop using your product. Integration testing shows you how different components in your product work together seamlessly.
Functional testing — focuses on testing the fulfillment of your customer requirements on your product. For example, if uploading images and editing their information before posting to others on your platform is one of the requirements, you will test and make sure such a requirement is fulfilled on your product using fewer steps. This is performed by the software developer at an early stage, and then perhaps you (or your technical lead or product manager) and then select non-technical potential customers.
Performance testing — checks the behavior of your software product when it is under significant load. Although you may not expect millions of users to use your product during the early stage, it is not a bad idea to test how your product performs when millions of customers are using your product. If you are not that ambitious, simulate at least ten thousand users using your product simultaneously.
Performance testing will reveal at what point your product becomes less responsive and even stop working. Knowing its limit will help you plan to improve your product before or after its release to your early adopters. Later, you need to conduct performance tests before launching your marketing campaigns or growth hacking.
Security testing — often overlooked, testing for security vulnerabilities will give you and your customers peace of mind. Your site will become a target if the value proposition is important enough to bring hundreds of customers in a short period. A rise in popularity makes it a target for hackers or even your competitors. Later in this article, we cover security testing in-depth and provide some recommendations. For now, you need to perform Static code analysis and vulnerability assessments before releasing your product to your customers — especially if it collects personal information and works online.
Test automation
Testing seems to be daunting, and often it is paramount to delivering value to your customers. However, testing automation helps you reduce the amount of manual testing you have to perform and maintain consistency in testing all the basics after each change in your software product — even the minor ones.
These are the testing that can be automated to a greater extent, relieving you of major headaches:
Unit testing
User interface testing (limited automation)
Integration testing
Performance testing
Security testing (static code analysis and vulnerability assessments)
We discuss automation in the later articles when we discuss the technology part of our blueprint.
Closure
We discussed different types of testing and then detailed prototype testing and software testing. Software testing is complex and, if not done right, can rapidly lose your customers. However, despite the complexity in testing your software, automation has made repeated complex testing possible, leaving very little manual testing for each release of your software.