Go package to provide delta coverage between your tests.

  • By Broadwing
  • Last update: Sep 30, 2022
  • Comments: 1

deltacoverage

Go package to provide delta coverage between your tests.

Installing

go install github.com/broadwing/deltacoverage/cmd/[email protected]

Example

$ deltacoverage
TestParseCoverageResult_ErrorsIfGivenNoTests 1.0%
TestParseCoverageResult_ReturnsCorrectValueGivenTestScriptTestCoverResult 3.2%
TestParseCoverageResult_ReturnsCorrectValueGivenNoTestScriptsTestCoverResult 0.0%
TestParseListTests_ReturnsZeroTestsIfGivenNoTests 0.0%
TestParseListTests_ReturnsCorrectValuesGivenTestListResult 0.0%
TestTestScript_ 54.2%

The test TestParseCoverageResult_ErrorsIfGivenNoTests contributes to exclusive 1.0% of the total coverage while TestScript_ is responsible for 54.2%.

Motivation

Herb Derby came up with this metric of “delta coverage”. 
You look at your test suite and you measure what coverage each test adds uniquely that no other test provides.
If you have lots of tests with no delta coverage, so that you could just delete them and not lose your 
ability to exercise parts of the system, then you should delete those tests, unless they have some some 
communication purpose.
—Kent Beck, “Is TDD Dead?”

Thank you @bitfield for the suggestion.

Roadmap

  • POC ✅
  • Brute force implementation using go test commands ✅
  • Get same metrics using coverprofiles of each test instead of brute force. It's going to improve performance from O(n2) to O(n) -> WIP
  • Borrow cover.go code and create deltagecoverage.go
  • (Dream) Implement as Go CLI feature

Download

deltacoverage.zip

Comments(1)

  • 1

    Improve performance from O(n2) to O(n)

    This PR refactors the code completely. Now, instead of running go test -run with a different combination of tests, we are generating one cover profile go test -coverprofile [temporary file] -run filtering only the specific test at a time, and then we process the cover profiles to get the same metrics.

    It has been shown to cut the total time to run in half for most cases but for some CPU-intensive tests, it decreased drastically. For example, to run for the package strconv in the std library, on my machine, the total time decreased from 60 secs to 15 sec.

Popular Tag

Related