SystemTestingTools
Stubbing tool for HTTP calls to allow more comprehensive + deterministic tests
Install / Use
/learn @AlanCS/SystemTestingToolsREADME
<img align="right" src="https://i.imgur.com/DdoC5Il.png" width="100" />
SystemTestingTools (for .net core 3.1+) extends your test capabilities, providing ways to create / return stubs, allowing you to run more comprehensive / deterministic / reliable tests in your local dev machine / build tool and in non-prod environments.
- supports interception of Http (HttpClient or WCF) calls:
- before they are sent, returning stubs (ideal for automated testing)
- after they are sent, where you can save the request and response (recording), log appropriately or replace bad responses by stubs (ideal for dev/test environments that are flaky or not ready)
- asserting outgoing calls (ie: making sure out downstream calls have SessionIds)
- intercept logs and run asserts on them
Recommended article to understand the philosophy behind T shaped testing, and how to best leverage this tool: https://www.linkedin.com/pulse/evolving-past-test-pyramid-shaped-testing-alan-sarli
In summary, you are better off using T shaped testing (focusing mainly component testing and using unit tests to complement those + contract testing) instead of the traditional testing pyramid, here is a summary of pros and cons:
| Benefit | Test pyramid | T shaped tests | Remediation | | ------------- |:-------------:|:-------------:| -----:| | Execution speed | :heavy_check_mark: | :heavy_check_mark: | | Ease of refactoring | :x: | :heavy_check_mark: | | Test all “your” code, like real users: more confidence | :x: | :heavy_check_mark: | | Documentation of external dependencies +++ | :x: | :heavy_check_mark: | | Balance between production code and testing code | :x: | :heavy_check_mark: | | Assert quality logs | :x: | :heavy_check_mark: | | Assert outgoing requests | :x: | :heavy_check_mark: | | Ease to achieve high test coverage | :x: | :heavy_check_mark: | | Document requirements for other teams (outsourcing, remote working) | :x: | :heavy_check_mark: | | Prepare monolith for later break out | :x: | :heavy_check_mark: | | Easier to TDD / BDD | :x: | :heavy_check_mark: | | Incentive for small methods / clean code | :heavy_check_mark: | :x: | PRs reviews, automated checks | Quickly find bugs | :heavy_check_mark: | :x: | Small commits | Handle shared states (cache, circuit breaker) | :heavy_check_mark: | :x: | Disable or use data so cache doesn’t matter
Basic capabilities for automated testing
You can use the extension method HttpClient.AppendHttpCallStub() to intercept Http calls and return a stub response, then use HttpClient.GetSessionLogs() and HttpClient.GetSessionOutgoingRequests() to get all the logs and outgoing Http calls relating to your session.
Simple example:
using SystemTestingTools;
using Shouldly; // nice to have :)
using Xunit;
[Fact]
public async Task When_UserAsksForMovie_Then_ReturnMovieProperly()
{
// arrange
var client = Fixture.Server.CreateClient(); // usual creating of HttpClient
client.CreateSession(); // extension method that adds a session header, should be called first
var response = ResponseFactory.FromFiddlerLikeResponseFile($"200_Response.txt");
var uri = new System.Uri("https://www.mydependency.com/api/SomeEndpoint");
client.AppendHttpCallStub(HttpMethod.Get, uri, response); // will return the stub response when endpoint is called
// act
var httpResponse = await client.GetAsync("/api/movie/matrix");
// assert logs (make sure the logs were exactly how we expected, no more no less)
var logs = client.GetSessionLogs();
logs.Count.ShouldBe(1);
logs[0].ToString().ShouldBe($"Info: Retrieved movie 'matrix' from downstream because it wasn't cached");
// assert outgoing (make sure the requests were exactly how we expected)
var outgoingRequests = client.GetSessionOutgoingRequests();
outgoingRequests.Count.ShouldBe(1);
outgoingRequests[0].GetEndpoint().ShouldBe($"GET https://www.mydependency.com/api/SomeEndpoint/matrix");
outgoingRequests[0].GetHeaderValue("User-Agent").ShouldBe("My app Name");
// assert return
httpResponse.ShouldNotBeNull();
httpResponse.StatusCode.ShouldBe(HttpStatusCode.OK);
var movie = await httpResponse.ReadJsonBody<MovieProject.Logic.DTO.Media>();
movie.ShouldNotBeNull();
movie.Id.ShouldBe("tt0133093");
movie.Name.ShouldBe("The Matrix");
movie.Year.ShouldBe("1999");
movie.Runtime.ShouldBe("136 min");
}
*We need the concept of sessions because many tests with many requests can be happening at the same time, so we need to keep things separated
Basic capabilities for stubbing in non prod environments
You can use the extension method IServiceCollection.InterceptHttpCallsAfterSending() to intercept Http calls (coming from HttpClient or WCF/SOAP), a lambda method will be called everytime for you to decide what to do; with a few helper methods to facilitate common requirements.
Simple example:
using SystemTestingTools;
public virtual void ConfigureServices(IServiceCollection services)
{
services.InterceptHttpCallsAfterSending(async (intercept) => {
if (intercept.Response?.IsSuccessStatusCode ?? false)
await intercept.SaveAsRecording("new/unhappy"); // save for later analysis
else
await intercept.SaveAsRecording("new/happy"); // save so it can be used to replace an unhappy response later (and analysis)
if (intercept.Response?.IsSuccessStatusCode ?? false)
return intercept.KeepResultUnchanged(); // if we got a happy response, just return the original, no need for stubs
var recording = RecordingCollection.Recordings.FirstOrDefault(
recording => recording.File.Contains("new/happy")
&& recording.Request.RequestUri.PathAndQuery == intercept.Request.RequestUri.PathAndQuery);
if (recording != null) // we found a happy response for the same endpoint, return it instead of original response
return intercept.ReturnRecording(recording, "unhappy response replaced by a happy one");
return intercept.KeepResultUnchanged(); // return original response
});
}
Automated testing setup
When creating your WebHostBuilder in your test project, to support HttpClient calls, add
.InterceptHttpCallsBeforeSending()
.IntercepLogs(minimumLevelToIntercept: LogLevel.Information,
namespaceToIncludeStart: new[] { "MovieProject" },
namespaceToExcludeStart: new[] { "Microsoft" });
Explanation: InterceptHttpCallsBeforeSending() will add a LAST DelegatingHandler to every HttpClient configured with services.AddHttpClient() (as recommended by Microsoft); so we can intercept and return a stub response, configured as above by the method AppendHttpCallStub().
InterceptLogs() allows namespaces to include or exclude in the logs, first the inclusion filter is executed, then the exclusion one. So if you configure namespaceToIncludeStart: new[] { "MovieProject" } namespaceToExcludeStart: new[] { "MovieProject.Proxy" }; then you will get all logs logged from classes whose namespace starts with MovieProject, except if they start with MovieProject.Proxy
Extra capabilities
1 - Multiple ways of creating responses
The HttpClient extension method AppendHttpCallStub() requires you to pass a HttpResponseMessage, which will be returned instead of making a real call downstream. You can create the response in a number of ways:
1.1: You can manually create a response:
var response = new HttpResponseMessage(HttpStatusCode.OK);
response.Headers.Add("Server", "Kestrel");
response.Content = new StringContent(@"{""value"":""whatever""}", Encoding.UTF8, "application/json");
1.2: Use ResponseFactory.From method, which loads the body from a string:
ResponseFactory.From(@"{""Message"":""some json content""}", HttpStatusCode.OK)
1.3: Use ResponseFactory.FromBodyOnlyFile method, which loads only the content of a file as the response body:
ResponseFactory.FromBodyOnlyFile($"response200.txt", HttpStatusCode.OK)
{"Message":"some json content"}
1.4: Use ResponseFactory.FromFiddlerLikeResponseFile method, which loads a file with the content just like Fiddler Web Debugger displays responses (with the optional added comments on the top):
ResponseFactory.FromFiddlerLikeResponseFile("response200.txt")
Which can load a response just like it's displayed in Fiddler:
HTTP/1.1 200 OK
Header1: some value
Header2: some other value
{"Message":"some json content"}
1.5: (RECOMMENDED WAY) Use ResponseFactory.FromRecordedFile method, which loads a file created by the recording feature bellow:
ResponseFactory.FromRecordedFile("response200.txt")
2 - Stub exceptions
You can test unhappy paths during Http calls and how well you handle them with the HttpClient.AppendHttpCallStub method exception overload.
// arrange
var client = Fixture.Server.CreateClient();
client.CreateSession();
var exception = new HttpRequestException("weird network error");
client.
