Non Functional Testing are testing aspects that are not related to our specific requirements but more of the quality of our software.

• Can our software scale?

• Is our software performant?

• Can it sustain load? What is the maximum load it can sustain?

• What do we do to extend our security? Are our servers secure?

As developers, we sometimes forget the importance of non-functional testing, but what good is an application that meets the requirements but cannot be online for more than an hour? Both are extremely important and we must dedicate time to each area of testing so our software not only meets the requirements, but has great quality!

In computer programming, unit testing is a procedure used to validate that individual units of source code are working properly. A unit is the smallest testable part of an application. In procedural programming a unit may be an individual program, function, procedure etc, while in object-oriented programming, the smallest unit is always a Class; which may be a base/super class, abstract class or derived/child class. Wikipedia

The testing process will take us through a journey of exploration of our software in order to determine its quality, if it met the gathered requirements, work as expected, and provide the required functionality. However, all the testing that we do is mostly controlled and under certain conditions. It is extremely difficult to determine all factors and all software conditions. Therefore attaining a testing nirvana is unrealistic.

Kaner, Cem; James Bach, Bret Pettichord (2001). Lessons Learned in Software Testing: A Context-Driven Approach. Wiley. p. 4. ISBN 0-471-08112-4.

It is believed that the earlier a bug is found the cheaper it is to fix it. This is something to remember as when things affect our income is when we become babies and really want to fix things then. So always take into consideration that bugs do cost money and we can be proactive about it. So what are some things apart from unit testing that we can do to improve the quality of our software? Well, apart from applying our functional and non-functional testing approaches we can also do some static and dynamic testing of our software.

Functional testing relies on the action of verification of specific requirements in our software. This can usually be tracked via use cases or Agile stories. "Can a user log in?", "Does the logout work", "Can I retrieve my password", "Can I purchase a book?". Our main focus of functional testing is that all these use cases need to be met.

On the other side of the spectrum, dynamic testing relies on the fact of executing test cases to verify software. Most of the time it can be the developers themselves or a QA team than can direct testing cases on executed code.

Static testing relies on human interaction not only with our code but with our developers and stakeholders. Some aspects can be code reviews, code/ui walkthroughs, design and modeling sessions, etc. Through experience, we can also attest that static testing is sometimes omitted or considered to be a waste of time. Especially when you need to hold up on production releases to review somebody else’s work. Not only that, but sometimes code reviews can spark animosity between developers and architects as most of the time the idea is to find holes in code, so careful tact must be in place and also maturity across development environments. In true fun we can say: “Come on, take the criticism, it is for the common good!”

This section is writtent for developers, so what is our focus? Our focus is to get awareness about the different aspects of testing and what are some of the testing focuses for the development phase:

• Unit Testing - We should be testing all the CFCs we create in our applications. Especially when dealing with Object Oriented systems, we need to have mocking and stubbing capabilities as well, as that might be the only way to test some CFCs.

• Integration Testing - Unit tests can only expose issues at the CFC level, but what happens when we put everything together? Integration testing in ColdBox allows you to test requests just like if they where executed from the browser. So all your application loads, interceptions, caching, dependency injection, ORM, database, etc, will fire in unison.

• UI Integration Testing - Testing verification via the UI using tools like Selenium is another great idea!

• Load Testing - There are tons of free load testing tools and they are easy to test how our code behaves under load. Don't open a browser with 5 tabs on it and start hitting enter! We have been there and it doesn't work that great :)

Here are some terms that we need to start getting familiar with:

• Test Plan: A test plan documents the strategy that will be used to verify and ensures that a product or system meets its design specifications and other requirements.

• Test Case: A set of conditions or variables under which a tester will determine whether an application or software system is working correctly or not. In our ColdFusion space, it might be represented by an MXUnit test case CFC.

• Test Script: A set of instructions that will be performed on the system under test to test that the system functions as expected. This can be manual or automated. Sometimes they can be called runners.

• Test Suite: A collection of test cases that are intended to be used to test a software program to show that it has some specified set of behaviors or met expectations.

• Test Data: A collection of pre-defined data or mocked data used to run against our test cases.

• Test Harness: A collection of software and test data configured to test a program unit by running it under varying conditions and monitoring its behavior and outputs.

• TDD: Test Driven Development - A software development process that relies on the repetition of a very short development cycle: first the developer writes a failing automated test case that defines a desired improvement or new function, then produces code to pass that test and finally refactors the new code to acceptable standards.

• BDD: BDD stands for behavior driven development and is highly based on creating specifications and expectations of results in a readable DSL (Domain Specific Language).

• Mock Object: Simulated objects that mimic the behavior of real objects in controlled ways.

• Stub Object: Simulated objects that have no behavior or implementations and can be controlled and injected with functionality.

• Assertions: The function to test if a predicate is true or false: X>1, 4=5

• Expectations : Much like assertions, it is closely linked to BDD to verify behavior.