Tips to successfully build your open-source projects #2. Test, test, and test again

5 min readOct 21, 2020

Testing is an important part of the software engineering process. And while it is already important in a professional environment, it is even more important for an open-source project.

Build confidence for yourself

One important subject I talked about in my two previous articles (Tips to successfully build your open-source projects #0. Why should you open-source your project? and Tips to successfully build your open-source projects #1. Don’t rush features, Code quality matters) is confidence when publishing your work to the open-source. Many people do not want to open-source their projects because they do not feel confident about their project and code quality.

Unit testing your project will help you be confident that your project is working as it should. This is very similar to building a new feature at work and be confident to deploy it to production because you spent some time unit testing your code and coordinating with the QA team to catch any issue ahead of the release.

Additionally, unit testing your code usually requires you to (re)design your code such that you can easily test it. Typically, you will end up breaking down your code into small and reusable functions. This translates into improved code quality, effectively increasing your confidence.

Build confidence for others

If your end-goal when open-sourcing your project is to get people to use it, then you will not only need to be confident in your project but also make sure that others are confident about it.

Before deciding to use your project, people will first investigate its quality and compare it with all possible alternatives to choose the most appropriate solution for their problem.

When comparing your project with some alternatives, there are some essential points of comparison: What are the supported features? How heavy is the project? How popular is it? How many issues are currently opened? Is it well tested? Is it well maintained?

While writing tests for your project will not make everyone use your project, it will definitely help.

Keep Control

You are building an open-source project on your personal time. When investing your own personal time on a project, one important rule is to control the time allocated to that project and make sure you do not become a slave to your own project.

It is already quite annoying to be on-call for a production issue at work, but at least you get paid for it. However, there is nothing worse than having reports of an issue on your open-source project and having to spend your weekend fixing it: there is not much fun in that and it can feel demotivating working for free on issues that could have been caught with unit tests early on.

When you are developing a new feature and testing it thoroughly, you are in control of the time dedicated to your project: if you don't have time, you can decide to do more testing later and just release your new feature at another time in the future.

But once your feature is released and a bug is reported by one of your users, you might feel the urge to immediately jump into troubleshooting it: at that point, you are no longer in control!

Of course, not all bugs reports are actually related to an issue in your project, and not all bug reports are critical. However, as a project maintainer, it may be hard to resist the temptation to take care of the issue, no matter how important it is. Thus, save yourself some time, and help yourself to remain in control, by testing as much as you can!

Help Adoption

People will not necessarily report the bugs they encountered.

New users might think this is an expected behavior, an unsupported feature, or a bug. They may not have the time to troubleshoot or to report the issue, and they will simply stop using your project and go for an alternative. This is especially true if they were investigating your project to see if it fits their needs.

Thus, if the issue affects some very basic features of your project, it can drastically affect the adoption of your project. Make sure that the core features of your project are properly tested, working as expected, and as little error-prone as possible.

Help Contributors

Last, but not least: unit tests make it easier to contribute to your project.

When people want to contribute, they usually just want to improve a specific behavior or fix a bug affecting them personally. In other words, very few contributors think about the project as a whole when submitting changes.

Only you know how your entire project works and you should not assume that others will. Contributors will just code the stuff they need and might break other existing components without noticing it.

Quick Tip: use CI

To wrap-up this article, I strongly advise you to set up a CI pipeline that automatically compiles your project and runs your tests whenever you push new code to the repository and whenever someone opens a pull request.

I personally use Travis, but there are plenty of alternatives such as Jenkins, Circle, or Shippable. Most of them are SaaS, usually free for open-source projects, easy to set up, and with a nice UI.

This can quickly save you a lot of time. For example, you can configure your CI to run your test on different operating systems, or with different language versions, helping you to guarantee the compatibility of your project on different platforms without spending too much time on it.