What is an unstable build?

Software management build systems like TeamCity or Jenkins provide automated build and testing on software team commits. If it cannot build the code ... well the build has failed as there are no binaries to test or even install. A complete no starter. But what if the binaries are built but a test failed? Is that a fail, or is that something less? Some build systems report a build with failing tests as "unstable". A useful clarification or a damaging depreciation of the build process?

What does "unstable" mean? Go on ... "make my day"?

Unstable tells the team that the team that the thing that did not pass (aka failed), does not really matter ... go on ignore it. Sure ... I'm busy ... I'll ignore anything that is not a fail.

That the code can be compiled to binaries (or whatever) is great. It means that the team has complied with the coding language but does not give any indication of what the code does. Useful, necessary, but not sufficient.

If the tests fail, is it useful to indicate it as 'unstable' rather than failed? Useful clarification of status?

So what does a test failure mean?

• System/application test failure

• The application has failed to meet a specification and therefore a requirement.

• Unit test failure

• A code element does not do what the original developer intended. Think Open Closed Principle.

If the application does not meet specification/requirement - that is a fail, not unstable.

If a code element has been altered outside of intent then then that is a fail, the application, or the code's maintainability, is compromised. That is not unstable, a developer has lowered code health and may well have broken the application.

Treat your tests like your application. A failure is a failure. "Unstable" just means ... "you may may be able to ignore".