You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
You can do it, but it is a bit involved - as Fatal calls os.Exit()
func testOsExit(t *testing.T, funcName string, testFunc func(*testing.T)) {
if os.Getenv(funcName) == "1" {
testFunc(t)
return
}
cmd := exec.Command(os.Args[0], "-test.run="+funcName)
cmd.Env = append(os.Environ(), funcName+"=1")
err := cmd.Run()
if e, ok := err.(*exec.ExitError); ok && !e.Success() {
return
}
t.Fatal("subprocess ran successfully, want non-zero exit status")
}
func TestxxxFailure(t *testing.T) {
testOsExit(t, "TestxxxFailure", func(t *testing.T) {
xxx(2)
})
}
What this does is patch the environment with a env variable matching the test function name, and then forks the test process running just that test function name again in the sub-process if the env variable is missing. If the env variable is there, then it knows it is running in the sub-proces and runs the actual test that should call Os.Exit with an error value.
It would be handy if testify could wrap this pattern somehow.
I'm not sure we want to make this a supported pattern. In my mind, if an error is "fatal", the program should terminate (which appears to be what the log package does), so it would be better to test those cases at a higher level (functional automation tests) which Testify isn't really designed for.
That's into philosophical arguments about testing. Subprocess testing has been part of go test lore since the start. The code above is adapted from a 2014 presentation by Andrew Gerrand. https://talks.golang.org/2014/testing.slide#1
Having said that, the subprocess test doesn't update the coverage data - so you can never get the 100% coverage you're looking for.
I'm not saying I have any objection to that kind of testing, just that Testify isn't really otherwise designed to support it, so this would represent an expansion of Testify's "scope". If that's where people want to go, that's fine with me.
although getting test coverage "credit" for unit tests that employ Gerrand's "BE_CRASHER" technique would be more satisfying, code annotations could help in code reviews, and it would be easy for reviewers to verify. Example: func MustBeGT3(x int) { if x < 4 { log.Fatal().Msg("x is < 4!; see ya")// covered in Test_using_Gerrand_MustBeGT3 } }
another helpful technique is to, in the app code, declare vars called logFatal and logWarn, and have the app code call logFatal instead of log.Fatal. So logFatal, which is normally in prod set to log.Fatal, can be set to log.Warn in unit tests. e.g.
// in X.go:
var logFatal = log.Fatal
var logWarn = log.Warn
func F() {
logFatal(...
}
// in test_X.go:
func Test_F(t *testing.T) {
old := logFatal
logFatal = logWarn
defer func() {
logFatal = old
}()
// various tests for func F()...
Activity
boyan-soubachov commentedon Dec 20, 2019
I'm not sure I understand, is the function being recursed?
Do you mind changing the example to be a bit more specific/realistic? I'd be happy to help out
NeilW commentedon Dec 23, 2019
You can do it, but it is a bit involved - as
Fatalcallsos.Exit()What this does is patch the environment with a env variable matching the test function name, and then forks the test process running just that test function name again in the sub-process if the env variable is missing. If the env variable is there, then it knows it is running in the sub-proces and runs the actual test that should call Os.Exit with an error value.
It would be handy if testify could wrap this pattern somehow.
glesica commentedon Dec 23, 2019
I'm not sure we want to make this a supported pattern. In my mind, if an error is "fatal", the program should terminate (which appears to be what the log package does), so it would be better to test those cases at a higher level (functional automation tests) which Testify isn't really designed for.
NeilW commentedon Dec 23, 2019
That's into philosophical arguments about testing. Subprocess testing has been part of
go testlore since the start. The code above is adapted from a 2014 presentation by Andrew Gerrand. https://talks.golang.org/2014/testing.slide#1Having said that, the subprocess test doesn't update the coverage data - so you can never get the 100% coverage you're looking for.
glesica commentedon Dec 23, 2019
I'm not saying I have any objection to that kind of testing, just that Testify isn't really otherwise designed to support it, so this would represent an expansion of Testify's "scope". If that's where people want to go, that's fine with me.
NefixEstrada commentedon Mar 18, 2020
Hello! I also use this funcionality. What I do, is create a custom type and embed
suite.Suitein it:After that, I just implement the functionality:
I also thing it would be a really nice addition to testify. After all, I do use it for unit testing
Goldziher commentedon Feb 25, 2022
This would be a very nice addition, I agree
wkdwyerNIH commentedon Dec 13, 2023
although getting test coverage "credit" for unit tests that employ Gerrand's "BE_CRASHER" technique would be more satisfying, code annotations could help in code reviews, and it would be easy for reviewers to verify. Example:
func MustBeGT3(x int) {if x < 4 {log.Fatal().Msg("x is < 4!; see ya")// covered in Test_using_Gerrand_MustBeGT3}}Antonboom commentedon Jan 8, 2024
Usually application with nice architecture have single exit point, in
package main:The rest code could be easily [moved in separate package and] tested without "Fatal catching".
👍
wkdwyerNIH commentedon Jan 9, 2024
another helpful technique is to, in the app code, declare vars called logFatal and logWarn, and have the app code call logFatal instead of log.Fatal. So logFatal, which is normally in prod set to log.Fatal, can be set to log.Warn in unit tests. e.g.