Expecto
A smooth testing lib for F#. APIs made for humans! Strong testing methodologies for everyone!
Install / Use
/learn @haf/ExpectoREADME
Expecto aims to make it easy to test CLR based software; be it with unit tests, stress tests, regression tests or property based tests. Expecto tests are parallel and async by default, so that you can use all your cores for testing your software. This also opens up a new way of catching threading and memory issues for free using stress testing.
With Expecto you write tests as values. Tests can be composed, reduced, filtered, repeated and passed as values, because they are values. This gives the programmer a lot of leverage when writing tests. Setup and teardown are just simple functions, no need for attributes.
Expecto comes with batteries included with an integrated test runner, but it's still open for extension due to its compositional model.
Expecto comes with performance testing, making statistically sound performance comparison simple.
Expecto also provides a simple API for property based testing using FsCheck.
Quickstart
dotnet new install "Expecto.Template::*"
dotnet new expecto -n PROJECT_NAME -o FOLDER_NAME
Follow Smooth Testing on YouTube to learn the basics.
What follows is the Table of Contents for this README, which also serves as the documentation for the project.
<!-- toc -->- Quickstart
- Installing
- IDE integrations
- .NET integration
- .NET support
- Testing "Hello world"
- Running tests
- Writing tests
- Expectations with
Expect main argsand command line – how to run console apps examples- Contributing and building
- BenchmarkDotNet usage
- You're not alone!
- Sending e-mail on failure – custom printers
- About test parallelism
- Migration notes
Installing
In your paket.dependencies:
nuget Expecto
nuget Expecto.BenchmarkDotNet
nuget Expecto.FsCheck
nuget Expecto.Hopac
Tests should be first-class values so that you can move them around and execute them in any context that you want.
Let's have look at what an extensive unit test suite looks like when running with Expecto:
IDE integrations
There's a NuGet Expecto.VisualStudio.TestAdapter for Visual Studio integration.
.NET integration
You can use dotnet run or dotnet watch from the command line.
dotnet watch -p MyProject.Tests run -f net6.0
Prettify stacktraces/ship test logs
To get a [complete logging solution][logary] and stacktrace highlighting, parsing and the ability to ship your build logs somewhere, also add these:
nuget Logary.Adapters.Facade prerelease
And in your tests:
open Hopac
open Logary
open Logary.Configuration
open Logary.Adapters.Facade
open Logary.Targets
[<EntryPoint>]
let main argv =
let logary =
Config.create "MyProject.Tests" "localhost"
|> Config.targets [ LiterateConsole.create LiterateConsole.empty "console" ]
|> Config.processing (Events.events |> Events.sink ["console";])
|> Config.build
|> run
LogaryFacadeAdapter.initialise<Expecto.Logging.Logger> logary
// Invoke Expecto:
runTestsInAssemblyWithCLIArgs [] argv
Now, when you use Logary in your app, you can see your log messages together with the log output/summary/debug printing of Expecto, and the output won't be interlaced due to concurrency.
TestResults file
Use --nunit-summary TestResults.xml or --junit-summary TestResults.junit.xml (JUnit support is incomplete).
.NET support
Expecto has its own .NET template! You could create a base .NET project with expecto. How to do that? Simply write following lines:
dotnet new install 'Expecto.Template::*'
dotnet new expecto -n PROJECT_NAME -o FOLDER_NAME
How to run it?
dotnet restore
dotnet run
Testing "Hello world"
The test runner is the test assembly itself. It's recommended to compile your test assembly as a console application. You can run a test directly like this:
open Expecto
let tests =
test "A simple test" {
let subject = "Hello World"
Expect.equal subject "Hello World" "The strings should equal"
}
[<EntryPoint>]
let main args =
runTestsWithCLIArgs [] args tests
No magic is involved here. We just created a single test and hooked it into the assembly entry point.
The Expect module contains functions that you can use to assert with.
A testing library without a good assertion library is like love without kisses.
Now compile and run! xbuild Sample.fsproj && mono --debug bin/Debug/Sample.exe
Running tests
Here's a simple test:
open Expecto
let simpleTest =
testCase "A simple test" <| fun () ->
let expected = 4
Expect.equal expected (2+2) "2+2 = 4"
Then run it like this, e.g. in the interactive or through a console app.
runTestsWithCLIArgs [] [||] simpleTest
which returns 1 if any tests failed, otherwise 0. Useful for returning to the operating system as error code.
It's worth noting that <| is just a way to change the associativity of the
language parser. In other words; it's equivalent to:
testCase "A simple test" (fun () ->
Expect.equal 4 (2+2) "2+2 should equal 4")
runTestsWithCLIArgs
Signature CLIArguments seq -> string[] -> Test -> int. Runs the passed tests
and also overrides the passed CLIArguments with the command line parameters.
runTestsWithCLIArgsAndCancel
Signature CancellationToken -> ExpectoConfig -> Test -> int. Runs the passed tests
and also overrides the passed CLIArguments with the command line parameters.
runTestsInAssemblyWithCLIArgs
Signature CLIArguments seq -> string[] -> int. Runs the tests in the current
assembly and also overrides the passed CLIArguments with the command line
parameters. All tests need to be marked with the [<Tests>] attribute.
runTestsInAssemblyWithCLIArgsAndCancel
Signature CancellationToken -> CLIArguments seq -> string[] -> int. Runs the tests in the current
assembly and also overrides the passed CLIArguments with the command line
parameters. All tests need to be marked with the [<Tests>] attribute.
Filtering with filter
You can single out tests by filtering them by name (e.g. in the interactive/REPL). For example:
open Expecto
open MyLib.Tests
integrationTests // from MyLib.Tests
|> Test.filter defaultConfig.joinWith.asString (fun z -> (defaultConfig.joinWith.format z).StartsWith "another test" ) // the filtering function
|> runTestsWithCLIArgs [] [||]
Shuffling with shuffle
You can shuffle the tests randomly to help ensure there are no run order dependencies. For example:
open Expecto
open MyLib.Tests
myTests // from MyLib.Tests
|> Test.shuffle defaultConfig.joinWith.asString
|> runTestsWithCLIArgs [] [||]
Stress testing
Tests can also be run randomly for a fixed length of time. The idea is that this will catch the following types of bugs:
- Memory leaks.
- Threading bugs running same test at same time.
- Rare threading bugs.
- Rare property test fails.
The default config will run FsCheck tests with a higher end size than normal.
