I recently became aware of an unexpected problem with garbage collection for serverless functions on AWS Lamba in dotnet core (this probably applies to other cloud providers with serverless offerings too). Luckily there’s a simple fix for it!

The problem arises when a Lambda function is experiencing consistent invocations over extended periods of time. When this happens, all the concurrent instances of the function will be needed, and workers will only rarely be recycled. In such a scenario we rely on the garbage collector to remove unused resources and free up memory. If this doesn’t happen the worker will eventually run out of memory and crash.

Usually developers do not need to concern themselves with the garbage collector. That is why they are made after all: to simplify memory management. However, it is possible that the garbage collector might not work as well as it should when running inside a serverless function. To fix it we need to understand a bit more about the garbage collector in dotnet.

There are 2 types of garbage collection in dotnet core: Workstation GC and Server GC. Server GC is optimized for server usage, under the assumption that CPU usage is more highly prioritized than memory. This type of garbage collection allows for much higher memory usage, which might be no problem on a big server, but for Lambda functions where memory is very often scarce (we are paying for it after all!) this might not always be the best approach. Workstation GC can in many cases be more performant when memory is limited, such as for serverless functions or docker containers with limited resources available.

It is worth noting that Server GC is the default garbage collector for ASP.NET core apps.

Take a look at some example charts from Microsoft

Server GC — Higher memory consumption and fewer garbage collections (the triangles)

Workstation GC — Lower memory consumption and frequent garbage collections

The difference here is quite large; Server GC reaches up to 300 MB memory consumption before collection, compared to the <10MB of Workstation GC.

Luckily the fix here is easy, simply add the following property to your .csproj file for the lambda:

  <PropertyGroup> 
	    <ServerGarbageCollection>false</ServerGarbageCollection>
	  </PropertyGroup>

As always, your milage may vary with this depending on your application, but this should yield a much more stable memory consumption on your lambda functions over time. This will lead to less workers crashing and therefore less cold starts.

#memory-improvement #aws #serverless #garbage-collection #dotnet-core