Docker — writing a smaller image with multi stage builds for. NET core.

Docker — writing a smaller image with multi stage builds for. NET core.

Docker — writing a smaller image with multi stage builds for. NET core. - I’ve been using docker for playing around with my dinky website, but the DockerFile/image has always been a bit brute forcey. It’s time to explore a somewhat more effective DockerFile!

Docker — writing a smaller image with multi stage builds for. NET core. - I’ve been using docker for playing around with my dinky website, but the DockerFile/image has always been a bit brute forcey. It’s time to explore a somewhat more effective DockerFile!

Docker Overview

Docker is a method of building applications/infrastructure/code within a container; a container being a self contained piece of software with all dependencies needed to run an application.

Though not directly related to a build server, they do have some overlap in some of the problems they try to solve. When utilizing either docker or a build server, your build process and its dependencies need to be codified… in code. The idea is that you’re writing “docker code” in order to describe the steps to build and deploy your app. This is very similar to using a build server in that you can be sure that any developer or server will be able to build or run your application code, without the hassle of installing all of your applications dependencies, as those dependencies are referenced within the docker “code” itself. (Note, you still need to have docker installed, and there are likely a few other caveats, especially when it comes to injecting variables into your docker containers.)

Current Image

The current image I’m using is quite small (code length wise), and due to that fact builds take longer than they should. This is due simply to the fact there are no real “checkpoints” in my build process. I’ll try to explain more about that while walking through my base image:

dnc2.1.401-v1-base

FROM microsoft/dotnet:2.1.401-sdk-stretch
WORKDIR /app
# Perform updates, install gnupg and sudo
RUN apt-get update \
    && apt-get -y upgrade \
    && apt-get -y dist-upgrade \
    && apt-get install -y gnupg \
    && apt-get install -y sudo

dnc2.1.401-v1-node

FROM kritner/builddotnetcore:dnc2.1.401-v1-base
WORKDIR /app
# Install node
RUN curl -sL deb.nodesource.com/setup_10.x | sudo -E bash - \
    && apt-get install -y nodejs

KritnerWebsite.DockerFile

FROM kritner/builddotnetcore:dnc2.1.401-v1-node
WORKDIR /app
# Copy everything to prep for build
COPY . ./
WORKDIR /app/src/KritnerWebsite.Web
# Publish code
RUN dotnet publish -c Release -o out
CMD dotnet out/KritnerWebsite.Web.dll

Issues with Current Image
  • Not really using “multi stage builds” (multiple “from” statements). I’m using a few different images, but it’s all being rolled up into the final image. This means I’m running a final image with a whole lot more “stuff” than what should be needed.
  • Due to the way I’m building my final KritnerWebsite.Dockerfile based off of my other images, it’s not very flexible when it comes to upgrading which sdk I’m using. I currently need to update dnc2.1.401-v1-base, rebuild dnc2.1.401-v1-node, then rebuild my actual website image.
  • Though related to the previous two points, I thought it deserved its own: I’m currently installing a LOT on top of the dnc image — things like sudo, node, performing OS level updates. Working with “separate” images for building dotnet core code, and running dotnet core, would help avoid some of this.

Refactoring my DockerFile

A Better Image Template (Thanks GaProgMan)

GaProgMan has worked a bit with docker, and had a few tips for me with a multi-step build process he gave me a few months ago for reference (yes, I’m just getting to this now):

## Assuming we are building for 2.1.300
## Change this value to match the version of the SDK
## found in the global.json
FROM microsoft/dotnet:2.1.300-sdk-alpine AS build
## Set the default build configuration to be Development
## Override this by adding a --build-arg switch to the call
## to docker build. i.e:
##   docker build . --file UI.dockerfile --tag projname-ui --build-arg target_configuration=Release
## Will build this project in Release rather than Development
ARG target_configuration=Development
WORKDIR /build
# Copy all the sln and csproj files, then run a restore. The .NET Core SDK
# doesn't need access to the source files (other than these) in order to
# restore packages.
# By doing this first, docker can cache the result of the restore. This is
# great for build times, because restore actions can take a long time.
COPY ./src/proj.name/proj.name.csproj proj.name.csproj
# Do the above for all of your csprojs
RUN dotnet restore
# This copy relies on the .dockerignore file listing bin and obj directories.
# If these aren't listed, then the generated project.assets.json files will
# be overwritten in this copy action - this will lead to us needing to run
# another restore.
# This, and all other copy commands, will follow any guidance supplied in
# our .dockerignore file. This file ensures that we only copy files from given
# directories or of given file types - it is similar in structure and usage to the
# .gitignore file
COPY ./src/proj.name .
COPY ./global.json ./global.json
RUN dotnet build --configuration ${target_configuration}  --no-restore
# FROM build AS publish
RUN dotnet publish --configuration ${target_configuration} --output "../dist" --no-restore --no-build
# Install all of the npm packages as a cache-able layer. Similar to when we did
# a dotnet restore, it will be skipped if npm packages never change.
# The install step is performed in the internal-npm-image container, the steps
# from which are run just-in-time in our down stream container (i.e this one)
WORKDIR /build
FROM internal-npm-image as webpack
COPY --from=build ./build/ClientApp ./ClientApp/
COPY --from=build ./build/webpack-config ./webpack-config/
COPY --from=build ./build/tsconfig.json ./build/tsconfig.aot.json ./build/package.json ./build/webpack.config.js ./
RUN npm run webpack-production
FROM microsoft/dotnet:2.1.0-aspnetcore-runtime-alpine as App
## Set the default runtime environment to be development.
## This can be overridden by providing a value via the --build-arg switch.
## For example:
##   docker build . --file UI.dockerfile --tag projname-ui --build-arg target_configuration=Release --build-arg target_env=Staging
## Will build as release, but with the Staging environment
ARG target_env=Development
## We have to "recreate" it here, because an ARG only exists within the
## context of a base image.
## So the version of target_env at the top of this dockerfile only exists
## within the "build" image and this one (which exists only wihtin the
## "App" image), is completely different to the earlier one.
WORKDIR /App
COPY --from=build ./dist ./
COPY --from=webpack ./tmp/wwwroot/ ./wwwroot/
ENV ASPNETCORE_URLS [http://+:5001](http://+:5001)
ENV ASPNETCORE_ENVIRONMENT="${target_env}"
EXPOSE 5001
ENTRYPOINT ["dotnet", "projname-ui.dll"]

Adapting the template to my build

I don’t want to copy exactly off of GaProgMan’s sample, luckily he commented it very well, so I’d know what’s happening. The most important thing I’m shooting for is creating more layers. These layers are important for ensuring more things will be cached; so not rebuilt (necessarily) with every build of the DockerFile.

First things first — I know I can cut down on my image size by utilizing two separate base images throughout the docker file:

  • Not really using “multi stage builds” (multiple “from” statements). I’m using a few different images, but it’s all being rolled up into the final image. This means I’m running a final image with a whole lot more “stuff” than what should be needed.
  • Due to the way I’m building my final KritnerWebsite.Dockerfile based off of my other images, it’s not very flexible when it comes to upgrading which sdk I’m using. I currently need to update dnc2.1.401-v1-base, rebuild dnc2.1.401-v1-node, then rebuild my actual website image.
  • Though related to the previous two points, I thought it deserved its own: I’m currently installing a LOT on top of the dnc image — things like sudo, node, performing OS level updates. Working with “separate” images for building dotnet core code, and running dotnet core, would help avoid some of this.

Previously, I was using only the SDK, which blows up my final image size by quite a bit — my images’ current size is 2.23 GB as per docker images (yeesh!).

So for the two images — sdk and runtime:

FROM microsoft/dotnet:2.2-aspnetcore-runtime AS base
RUN apt-get update \
    && apt-get -y upgrade \
    && apt-get -y dist-upgrade \
    && apt-get install -y gnupg \
    && apt-get install -y sudo \
    && curl -sL deb.nodesource.com/setup_10.x | sudo -E bash - \
    && apt-get install -y nodejs
FROM microsoft/dotnet:2.2-sdk AS build
RUN apt-get update \
    && apt-get -y upgrade \
    && apt-get -y dist-upgrade \
    && apt-get install -y gnupg \
    && apt-get install -y sudo \
    && curl -sL deb.nodesource.com/setup_10.x | sudo -E bash - \
    && apt-get install -y nodejs

In the above we’re running a few commands on the base images for the purpose of installing nodejs — which we’ll need both for building and running the angular app; at least I’m pretty sure it’s needed for both right?

WORKDIR /src
COPY ["./src/KritnerWebsite.Web/KritnerWebsite.Web.csproj", "src/KritnerWebsite.Web/KritnerWebsite.Web.csproj"]
RUN dotnet restore "src/KritnerWebsite.Web/KritnerWebsite.Web.csproj"

Next, we’ll do the dotnet restore on the single copied project file — the reasoning behind this was pretty well explained in the above example, but I didn’t really realize it worked this way until seeing it in GaProMan’s comments. Basically, this restored “layer” can be cached, and never “rebuilt” unless something in the dependencies changes, saving on time when rebuilding our docker image!

COPY ["./src/KritnerWebsite.Web/ClientApp/package.json", "src/KritnerWebsite.Web/ClientApp/package.json"]
RUN cd src/KritnerWebsite.Web/ClientApp \
    && npm install

Same idea in the above, but for npm packages instead of .net dependencies.

COPY ["src/KritnerWebsite.Web/", "src/KritnerWebsite.Web"]
WORKDIR /src/src/KritnerWebsite.Web
RUN dotnet build -c Release -o /app --no-restore

In the above, I’m copying the entirety of the buildable source directory, and performing a build with the .net CLI. Special note that the --no-restore option is being used as a restore operation was performed previously.

FROM build AS publish
RUN dotnet publish -c Release -o /app --no-restore --no-build

Here, in a similar idea to the build layer, we’re performing a publish; making sure not to restore or build as both have already been completed.

Finally:

FROM base AS final
WORKDIR /app
COPY --from=publish /app .
ENTRYPOINT ["dotnet", "KritnerWebsite.Web.dll"]

In the above we’re copying our built application from the publish image, into a new “final” image that was based off of “base” (the run time).

The new DockerFile

The new DockerFile looks like this in its entirety:

# docker build -t kritner/kritnerwebsite .
# docker run -d -p 5000:5000 kritner/kritnerwebsite
# docker push kritner/kritnerwebsite
# Runner image - Runtime + node for ng serve
FROM microsoft/dotnet:2.2-aspnetcore-runtime AS base
RUN apt-get update \
    && apt-get -y upgrade \
    && apt-get -y dist-upgrade \
    && apt-get install -y gnupg \
    && apt-get install -y sudo \
    && curl -sL deb.nodesource.com/setup_10.x | sudo -E bash - \
    && apt-get install -y nodejs
# Builder image - SDK + node for angular building
FROM microsoft/dotnet:2.2-sdk AS build
RUN apt-get update \
    && apt-get -y upgrade \
    && apt-get -y dist-upgrade \
    && apt-get install -y gnupg \
    && apt-get install -y sudo \
    && curl -sL deb.nodesource.com/setup_10.x | sudo -E bash - \
    && apt-get install -y nodejs
WORKDIR /src
# Copy only the csproj file(s), as the restore operation can be cached, 
# only "doing the restore again" if dependencies change.
COPY ["./src/KritnerWebsite.Web/KritnerWebsite.Web.csproj", "src/KritnerWebsite.Web/KritnerWebsite.Web.csproj"]
# Run the restore on the main csproj file
RUN dotnet restore "src/KritnerWebsite.Web/KritnerWebsite.Web.csproj"
# Contains the angular related dependencies, similar to csproj above result is cachable.
COPY ["./src/KritnerWebsite.Web/ClientApp/package.json", "src/KritnerWebsite.Web/ClientApp/package.json"]
# Install the NPM packages
RUN cd src/KritnerWebsite.Web/ClientApp \
    && npm install
# Copy the actual files that will need building
COPY ["src/KritnerWebsite.Web/", "src/KritnerWebsite.Web"]
WORKDIR /src/src/KritnerWebsite.Web
# Build the .net source, don't restore (as that is its own cachable layer)
RUN dotnet build -c Release -o /app --no-restore

FROM build AS publish
# Perform a publish on the build code without rebuilding/restoring. Put it in /app
RUN dotnet publish -c Release -o /app --no-restore --no-build
# The runnable image/code
FROM base AS final
WORKDIR /app
COPY --from=publish /app .
ENTRYPOINT ["dotnet", "KritnerWebsite.Web.dll"]

Now that the image is built, I can run it like normal to test it out:

docker run -d -p 5000:5000 kritner/kritnerwebsite

Huh, it actually seems to have worked! :D

Now I can push the image up to dockerhub, and pull it down on my server.

docker push kritner/kritnerwebsite

Now, to see the difference in size between the previous image and the current, I run docker images and am presented with:

So we went from a chonky 2.23GB to a cool 417MB, nice!

Wrap Up

Thanks to GaProgMan for pointing me in the right direction for making my docker image more useful. Code for this post can be found:

==================================

Thanks for reading :heart: If you liked this post, share it with all of your programming buddies! Follow me on Facebook | Twitter

Learn More

☞ Docker Mastery: The Complete Toolset From a Docker Captain

☞ Docker and Kubernetes: The Complete Guide

☞ Docker for the Absolute Beginner - Hands On - DevOps

☞ Docker Crash Course for busy DevOps and Developers

☞ The Docker for DevOps course: From development to production

☞ Docker for Node.js Projects From a Docker Captain

☞ Docker Certified Associate 2019

WordPress in Docker. Part 1: Dockerization

WordPress in Docker. Part 1: Dockerization

This entry-level guide will tell you why and how to Dockerize your WordPress projects.

This entry-level guide will tell you why and how to Dockerize your WordPress projects.

How to debug an ASP.NET Core Docker container in Windows AND Linux

How to debug an ASP.NET Core Docker container in Windows AND Linux

How to debug an ASP.NET Core Docker container in Windows AND Linux - Docker is an revolutionary tool that has far too many benefits to list in this blog post. Instead, we will be walking through the tooling that can be leveraged when you create and debug your next ASP.NET Core Docker application.

There is no question that .NET Core has exploded in popularity over the last couple of years. The new cross-platform successor to the .NET Framework has opened many new doors to developers. Using tools such as Docker have allowed developers to deploy their solutions in very repeatable and reliable ways.

Taking it a step further, what would be a better way to highlight the cross platform nature of .NET Core than exploring this on Windows AND Linux!

VISUAL STUDIO TOOLS FOR DOCKER

On the Windows side of the house, I have been impressed by the tooling that exists natively in Visual Studio. Using Visual Studio Tools for Docker makes working with containers a breeze. You can see this firsthand when creating a new ASP.NET Core project and selecting the Enable Docker Support checkbox.

Once complete, you will notice a new Dockerfile is created at the root of the project (shown below). The Dockerfile will run your application inside of a container however, that’s not all. To guarantee consistency across development machines, you can see the Dockerfile actually restores and publishes the project inside a build container. Say goodbye to those “it builds on my machine” excuses!

FROM microsoft/dotnet:2.1-aspnetcore-runtime AS base
WORKDIR /app
EXPOSE 80
EXPOSE 443

FROM microsoft/dotnet:2.1-sdk AS build
WORKDIR /src
COPY ["JrTech.Docker.Web/JrTech.Docker.Web.csproj", "JrTech.Docker.Web/"]
RUN dotnet restore "JrTech.Docker.Web/JrTech.Docker.Web.csproj"
COPY . .
WORKDIR "/src/JrTech.Docker.Web"
RUN dotnet build "JrTech.Docker.Web.csproj" -c Release -o /app

FROM build AS publish
RUN dotnet publish "JrTech.Docker.Web.csproj" -c Release -o /app

FROM base AS final
WORKDIR /app
COPY --from=publish /app .
ENTRYPOINT ["dotnet", "JrTech.Docker.Web.dll"]

Auto generating a Dockerfile is a great first step however, your development process is going to be rocky if you cannot debug your code. Luckily Visual Studio does some magic for us to allow just that. If you run the application, the following output will be shown in the debug window. This highlights how Visual Studio is running the Docker container with the remote debugger attached.

docker run -dt -v "C:\Users\jason\vsdbg\vs2017u5:/remote_debugger:rw" -v "C:\Jason\Repositories\gc\ahbc_dotnet_201810\JrTech.Docker.Web\JrTech.Docker.Web:/app" -v "C:\Users\jason\AppData\Roaming\ASP.NET\Https:/root/.aspnet/https:ro" -v "C:\Users\jason\AppData\Roaming\Microsoft\UserSecrets:/root/.microsoft/usersecrets:ro" -v "C:\Users\jason.nuget\packages:/root/.nuget/fallbackpackages2" -v "C:\Program Files\dotnet\sdk\NuGetFallbackFolder:/root/.nuget/fallbackpackages" -e "DOTNET_USE_POLLING_FILE_WATCHER=1" -e "ASPNETCORE_ENVIRONMENT=Development" -e "ASPNETCORE_URLS=https://+:443;http://+:80" -e "ASPNETCORE_HTTPS_PORT=44372" -e "NUGET_PACKAGES=/root/.nuget/fallbackpackages2" -e "NUGET_FALLBACK_PACKAGES=/root/.nuget/fallbackpackages;/root/.nuget/fallbackpackages2" -p 56567:80 -p 44372:443 --entrypoint tail jrtechdockerweb:dev -f /dev/null

We can now set breakpoints and debug the application while it is running. Many of us may take this for granted however, we can see Visual Studio had to do a little bit of work to put this together for us.

I have to say, I’ve been impressed with how easy it to create an ASP.NET Core Docker container with Visual Studio. What could make it even better? Doing the same on a Linux OS! Next we will go through the same exercise with VSCode on Linux.

VISUAL STUDIO CODE ON LINUX

One of the best things about .NET Core is that it is completely cross platform. It’s been a couple years but, it still feels weird to say… we can develop, build, and run ASP.NET Core on the operating system of our choice. Visual Studio Code is also cross platform and can be run on Windows, Mac, or Linux.

To get started, we need to install the following components.

DOCKER VISUAL STUDIO CODE EXTENSION

Visual Studio Code has a great extension subsystem so, it is no surprise that there is Docker extension readily available.

Once we have the extension installed, we can get started by creating our new project. Visual Studio Code is designed for working with all kinds of languages and frameworks so, it is no surprise that we don’t have a “new project template” available. No need to worry, .NET Core has an awesome command line interface!

Using the dotnet new command, we can create a new ASP.NET Core MVC project.

[email protected]:~/Source$ dotnet new mvc -n JrTech.Docker.Web -o JrTech.Docker.Web
The template "ASP.NET Core Web App (Model-View-Controller)" was created successfully.
This template contains technologies from parties other than Microsoft, see https://aka.ms/aspnetcore-template-3pn-210 for details.

Processing post-creation actions...
Running 'dotnet restore' on JrTech.Docker.Web/JrTech.Docker.Web.csproj...
Restoring packages for /home/jason/Source/JrTech.Docker.Web/JrTech.Docker.Web.csproj...
Generating MSBuild file /home/jason/Source/JrTech.Docker.Web/obj/JrTech.Docker.Web.csproj.nuget.g.props.
Generating MSBuild file /home/jason/Source/JrTech.Docker.Web/obj/JrTech.Docker.Web.csproj.nuget.g.targets.
Restore completed in 531.14 ms for /home/jason/Source/JrTech.Docker.Web/JrTech.Docker.Web.csproj.

Restore succeeded.

[email protected]:~/Source$ code .

When Visual Studio Code launches, you will see the following popup on the bottom right corner of the IDE. Selecting ‘Yes’ will create a tasks.json file with the required build steps.

We should now see a new web project similar to the one we created with Visual Studio. There is one key difference though, we do not have a Dockerfile yet. This is where the Docker extension helps us out. We can add a Dockerfile using the Add Docker Files to Workspace command. For a default file, select ASP.NET Core as the application platform, Linux as the operating system, and port 80 as the default port.

Looking at it newly created Dockerfile, we can see a familiar file format. In essence, this is the same multistage Dockerfile we had with Visual Studio.

FROM microsoft/dotnet:2.2-aspnetcore-runtime AS base
WORKDIR /app
EXPOSE 80

FROM microsoft/dotnet:2.2-sdk AS build
WORKDIR /src
COPY ["JrTech.Docker.Web.csproj", "./"]
RUN dotnet restore "./JrTech.Docker.Web.csproj"
COPY . .
WORKDIR "/src/."
RUN dotnet build "JrTech.Docker.Web.csproj" -c Release -o /app

FROM build AS publish
RUN dotnet publish "JrTech.Docker.Web.csproj" -c Release -o /app

FROM base AS final
WORKDIR /app
COPY --from=publish /app .
ENTRYPOINT ["dotnet", "JrTech.Docker.Web.dll"]

The next step is setting up our debug configuration so we can debug our application while it is running in a container. From the Debug pane, we can select Add Configuration from the drop-down to add our new configuration.

We have a ton of options to choose from but, in our case, we want to add the Docker: Launch .NET Core (Preview) configuration.

Depending on our containers requirements we can apply specific configurations here. With a newly created project, using the defaults works fine.

With our new configuration selected, clicking the play button will build and run our container with the debugger attached. As we did with Visual Studio, we can now add breakpoints and step through our application.

SUMMARY

As I’ve said previously, Docker is a very powerful technology, especially when combined with an orchestrator like Kubernetes. It is great to see the developer tooling we get out of the box with ASP.NET Core. I’ve been on a bit of a Linux kick lately so, it’s encouraging that we can replicate this behavior on multiple operating systems.

Happy coding!

Originally published by  JROB5756 at espressocoder.com

=============================

Thanks for reading

If you liked this post, share it with all of your programming buddies!

Follow me on Facebook | Twitter

How to build a Docker container for beginners

Tutorial Laravel 6 with Docker and Docker-Compose

Docker for the Absolute Beginner - Hands On - DevOps

Docker Basics: Docker Compose

Docker for Absolute Beginners


How to make a Windows Service from .Net Core 3.0

How to make a Windows Service from .Net Core 3.0

NET Core 3.0, it's a lot easier to create Windows Services: just a single line of code ... If the application runs on a Windows system, the method ..

In this blog post, we will create a demo Windows Service application which includes a set of features such as reading configurations, logging to files, dependency injection, file system watcher, and so on. The application will be written in .NET Core 3.0, which introduces new concepts like generic host, worker service, background service, and so on. We will go over the installation process of our Windows Service application as well.

The complete solution can be found in this GitHub repository. This application can be used as a bare-bones template for Windows Service applications or Console applications.

Why do we build Windows Service applications?

Microsoft Windows services allow us to create long-running executable applications that run in their own Windows sessions. Windows services don’t have any user interface, can be automatically started when the computer reboots, and can be paused and restarted.

Windows services are ideal for long-running functionality that does not interfere with other users who are working on the same computer. For example, in a Windows Service application, we can use a FileSystemWatcher to listen to the file system change notifications and raise events when a directory, or a file in a directory, changes. The beauty is that the Windows Service application handles all the events in background.

Practically, we usually run services in the security context of a specific user account that is different from the logged-on user or the default computer account. So a hacker cannot easily mess up the file system or the service related database through a compromised computer.

If you have created a Windows Service application in .NET framework, then you must remember the pain of debugging the Windows Service application. During those old days, the tool TopShelf helped us a little bit, but not much. Now, with the .NET Core 3.0, the experience of developing a Windows Service application is much more pleasant. In my opinion, the concept of a Windows Service is clearer as well.

In order to follow along, you need to have .NET Core 3.0 SDK installed. Also, you need to have Admin privilege in your computer or the hosting server, so that you can install the Windows Service and/or remove it.

Let’s first create a basic ASP.NET Core application and configure it to be able to be hosted in a Windows Service.

Create a bare-bones Windows Service application

We will use the worker service template from .NET Core as a starting point. If you are using Visual Studio, then you can follow the steps below:
(1) Create a new project.
(2) Select **Worker Service**. Select **Next**.
(3) Set the project name as “Demo”. Then select **Create**.
(4) In the **Create a new Worker service** dialog, select **Create**.

If you are using .NET CLI, then you can use the following command to create a solution which contains a Worker Service project.

dotnet new sln -o WindowsServiceDemo -n Demo
cd .\WindowsServiceDemo\
dotnet new worker -o Demo
dotnet sln add .\Demo\

In order to enable the worker service app to run as a Windows Service, we need to update the project a little bit by doing the following steps:

  1. Add a NuGet package [Microsoft.Extensions.Hosting.WindowsServices](https://www.nuget.org/packages/Microsoft.Extensions.Hosting.WindowsServices).
  2. Update the Program.cs by adding the IHostBuilder.UseWindowsService() extension method to the CreateHostBuilder process. The code snippet below shows an example.

Program.cs

public class Program
{
    public static void Main(string[] args)
    {
        CreateHostBuilder(args).Build().Run();
    }

    public static IHostBuilder CreateHostBuilder(string[] args) =>
        Host.CreateDefaultBuilder(args)
            .UseWindowsService()
            .ConfigureAppConfiguration((context, config) =>
            {
                // configure the app here.
            })
            .ConfigureServices((hostContext, services) =>
            {
                services.AddHostedService<Worker>();
            });
}

Add line 10 to make the worker service app to be able to host in a Windows Service

In the code above, line 10 is the key to creating a Windows Service app. When the application is hosted in a Windows Service, the extension method IHostBuilder.UseWindowsService() will set the ContentRoot, configure logging, set the host lifetime to WindowsServiceLifetime, and so on.

Voila~ Now we can build the app.

Through these simple steps, we have created an ASP.NET Core app that can be hosted in a Windows Service. Moreover, this app is automatically a Console application that we can run it directly via executing the Demo.exe file in the output folder. This setup allows us to debug the application as a Console app, and allows us to host the app in a Windows Service with minimum configurations.

Bonus: we can check if the app is running as a Windows Service or not using the bool WindowsServiceHelpers.IsWindowsService() method, which returns true if the current process is hosted as a Windows Service, otherwise false.

Add Serilog as a logging provider

Logging is essential for monitoring the status of our application. We definitely need logging for Windows Service applications, because they don’t have any interface and they are totally in the background. In this application, we will use Serilog to log messages to both Console output and physical files.

We will need to install the following NuGet packages that are related to Serilog: Serilog.Enrichers.Thread, Serilog.Extensions.Hosting, Serilog.Sinks.Console, and Serilog.Sinks.File. All of these NuGet packages should use their latest versions.

Then we update the Main method in the Program.cs file to be the code snippet below.

Program.cs

public static void Main(string[] args)
{
    const string loggerTemplate = @"{Timestamp:yyyy-MM-dd HH:mm:ss} [{Level:u4}]<{ThreadId}> [{SourceContext:l}] {Message:lj}{NewLine}{Exception}";
    var baseDir = AppDomain.CurrentDomain.BaseDirectory;
    var logfile = Path.Combine(baseDir, "App_Data", "logs", "log.txt");
    Log.Logger = new LoggerConfiguration()
        .MinimumLevel.Override("Microsoft", LogEventLevel.Warning)
        .Enrich.With(new ThreadIdEnricher())
        .Enrich.FromLogContext()
        .WriteTo.Console(LogEventLevel.Information, loggerTemplate, theme: AnsiConsoleTheme.Literate)
        .WriteTo.File(logfile, LogEventLevel.Information, loggerTemplate,
            rollingInterval: RollingInterval.Day, retainedFileCountLimit: 90)
        .CreateLogger();

    try
    {
        Log.Information("====================================================================");
        Log.Information($"Application Starts. Version: {System.Reflection.Assembly.GetEntryAssembly()?.GetName().Version}");
        Log.Information($"Application Directory: {AppDomain.CurrentDomain.BaseDirectory}");
        CreateHostBuilder(args).Build().Run();
    }
    catch (Exception e)
    {
        Log.Fatal(e, "Application terminated unexpectedly");
    }
    finally
    {
        Log.Information("====================================================================\r\n");
        Log.CloseAndFlush();
    }
}

public static IHostBuilder CreateHostBuilder(string[] args) =>
    Host.CreateDefaultBuilder(args)
        .UseWindowsService()
        .ConfigureAppConfiguration((context, config) =>
        {
            // Configure the app here.
        })
        .ConfigureServices((hostContext, services) =>
        {
            services.AddHostedService<Worker>();
        })
        .UseSerilog();

add Serilog

In the code snippet above, we added two logging sinks: (1) Console (line 10) with color theme, and (2) plain text files (line 11) that are rolling every day. The logging message are enriched with ThreadID and LogContext, which are two common fields that can help us diagnosing issues if any.

In the end, we add the line 44, .UseSerilog(), to the HostBuilder, so that the host will use Serilog as a logging provider.

Caveat:
By default, the log file path (in line 11) should be able to use a relative path with respect to the assembly entry file. However, when an application is hosted in a Windows Service, the current working directory is set to the “ _C:\WINDOWS\system32_” folder, which is not a good place for log files. So I used an absolute path, with respect to AppDomain.CurrentDomain.BaseDirectory, to make sure the log files are written and saved into the proper location.

Now, if we run the application, we should be able to see both Console outputs and a log file with all logging messages.

We can implement the Windows Service app to run scheduled background tasks, execute long running jobs, and so on. In this blog post, we will utilized a FileSystemWatcher to run background tasks when some specific file system events raise. This process is useful to monitor shared network folders or SFTP folders, in which users drop files.

Add FileSystemWatcher

For those who are new to FileSystemWatcher, you can read more from this article in Microsoft Docs. We are going to add a FileSystemWatcher to listen to the file system change notifications when a new txt file is created in a directory, C:\temp.

The FileSystemWatcher is better to live in the Worker service, because they will have the same lifetime. We initialize the FileSystemWatcher when the Worker service starts, and we dispose the FileSystemWatcher when the Worker service disposes.

The code snippet below shows an example Worker.cs file.

Worker.cs

public class Worker : BackgroundService
{
    private readonly ILogger<Worker> _logger;
    private FileSystemWatcher _folderWatcher;
    private readonly string _inputFolder;

    public Worker(ILogger<Worker> logger)
    {
        _logger = logger;
        _inputFolder = @"C:\temp";
    }

    protected override async Task ExecuteAsync(CancellationToken stoppingToken)
    {
        await Task.CompletedTask;
    }

    public override Task StartAsync(CancellationToken cancellationToken)
    {
        _logger.LogInformation("Service Starting");
        if (!Directory.Exists(_inputFolder))
        {
            _logger.LogWarning($"Please make sure the InputFolder [{_inputFolder}] exists, then restart the service.");
            return Task.CompletedTask;
        }

        _logger.LogInformation($"Binding Events from Input Folder: {_inputFolder}");
        _folderWatcher = new FileSystemWatcher(_inputFolder, "*.TXT")
        {
            NotifyFilter = NotifyFilters.CreationTime | NotifyFilters.LastWrite | NotifyFilters.FileName |
                              NotifyFilters.DirectoryName
        };
        _folderWatcher.Created += Input_OnChanged;
        _folderWatcher.EnableRaisingEvents = true;

        return base.StartAsync(cancellationToken);
    }

    protected void Input_OnChanged(object source, FileSystemEventArgs e)
    {
        if (e.ChangeType == WatcherChangeTypes.Created)
        {
            _logger.LogInformation($"InBound Change Event Triggered by [{e.FullPath}]");

            // do some work

            _logger.LogInformation("Done with Inbound Change Event");
        }
    }

    public override async Task StopAsync(CancellationToken cancellationToken)
    {
        _logger.LogInformation("Stopping Service");
        _folderWatcher.EnableRaisingEvents = false;
        await base.StopAsync(cancellationToken);
    }

    public override void Dispose()
    {
        _logger.LogInformation("Disposing Service");
        _folderWatcher.Dispose();
        base.Dispose();
    }
}

A background service with a file system watcher

In line 28, we set the filter to be “*.TXT”, which tells the FileSystemWatcher to watch for specify the type of files in the input folder, which is the txt file type in this project. The FileSystemWatcher accepts event handlers for Changed, Created, Deleted, and Renamed events. For demo purposes, we only handle the new txt file Created events in this project.

If we run the application, then we are able to observe the effects of the Worker service and the FileSystemWatcher. Once a new txt file is created in the C:\temp folder, the application will get notified and the Input_OnChanged event delegate will be called.

Awesome. Now we have a background process to watch the file changes.

Note: In order to work on files or folders in the file system, it’s important to make sure that the current user (for debugging purposes) and the Log On As account have proper permission to the intended working directory.

Bonus: We can use WindowsIndentity.GetCurrent().Name to verify the current user name. I usually write the user name to the application logs as a tracking measure.

Add configuration files

You might have noticed that the input folder path is a magic string, C:\temp, in the code above. We can improve the code by loading a configuration file to get the input folder path.

We add a JSON object “AppSettings” to the appsettings.json file. For demo purposes, we only add one property, InputFolder, in the AppSettings object. The settings can be extended as needed.

appsettings.json

"AppSettings": {
    "InputFolder": "C:\\temp"
}

In order to bind the AppSettings JSON value, we create a C# class file, AppSettings.cs, as follows.

AppSettings.cs

public class AppSettings
{
    public string InputFolder { get; set; }
}

Then, we register the configuration, AppSettings, in the Dependency Injection (DI) container by adding a line in the Program.cs file as follows.

Program.cs

.ConfigureServices((hostContext, services) =>
{
    services.AddHostedService<Worker>();
    services.Configure<AppSettings>(hostContext.Configuration.GetSection("AppSettings"));
})

In the end, we can inject the settings to the Worker service like the following code snippet.

Worker.cs

public Worker(ILogger<Worker> logger, IOptions<AppSettings> settings)
{
    _logger = logger;
    _inputFolder = settings.Value.InputFolder;
}

In this way, we eliminated the magic string, and we have a more extendable code base.

Caveat: The Hosting Environment
It is known that ASP.NET Core web app uses the environment variable ASPNETCORE_ENVIRONMENT to determine the web host environment. However, in the case of Generic Host (link), the host environment is determined by the environment variable DOTNET_ENVIRONMEN by default, which is different from the one for Web applications. Well, you can always overwrite the HostEnvironment using a key-value pair with the key “environment” and its value.

We don’t need to explicitly add JSON file providers for the appsettings.json and appsettings.{environment}.json files, because they are automatically configured inside the Host.CreateDefaultBuilder(args) method. So you might want to make sure the environment name is correctly set, if you want to load correct settings in the appsettings.{environment}.json file.

Add a Scoped Service

In many cases, our applications depend on some short-lived services, for example, database connections, HTTP Client. We don’t want the application holds unnecessary stale resources, so we register those short-lived services as Scoped or Transient dependencies in the DI container. All these should be straightforward in Web applications. However, in Windows Service Applications, there’s some extra work to do.

For demo purposes, we will keep this application simple, and we add two contrived services in the Demo project. The following two code snippets show the SeriveA and ServiceB classes.

ServiceA.cs

public interface IServiceA
{
    void Run();
}

public class ServiceA : IServiceA
{
    private readonly ILogger<ServiceA> _logger;
    private readonly IServiceB _serviceB;

    public ServiceA(ILogger<ServiceA> logger, IServiceB serviceB)
    {
        _logger = logger;
        _serviceB = serviceB;
    }

    public void Run()
    {
        _logger.LogInformation("In Service A");
        _serviceB.Run();
    }
}

ServiceB.cs

public interface IServiceB
{
    void Run();
}

public class ServiceB : IServiceB
{
    private readonly ILogger<ServiceB> _logger;

    public ServiceB(ILogger<ServiceB> logger)
    {
        _logger = logger;
    }

    public void Run()
    {
        _logger.LogInformation("In Service B");
    }
}

We inject the ServiceB into the ServiceA, and we will use ServiceA as an entry point to run the process when a new file is created and detected by the FileSystemWatcher in the Worker service.

We register the ServiceA and the ServiceB in the Program.cs file as follows.
Program.cs

.ConfigureServices((hostContext, services) =>
{
    services.AddHostedService<Worker>();
    services.Configure<AppSettings>(hostContext.Configuration.GetSection("AppSettings"));
    services.AddScoped<IServiceA, ServiceA>();
    services.AddScoped<IServiceB, ServiceB>();
})

Then we inject the IServiceA to the Worker service and call serviceA.Run() as follows.

Worker.cs

public class Worker : BackgroundService
{
    // ...
    private readonly IServiceA _serviceA;

    public Worker(ILogger<Worker> logger, IOptions<AppSettings> settings, IServiceA serviceA)
    {
        // ...
        _serviceA = serviceA;
    }
    // ...
    protected void Input_OnChanged(object source, FileSystemEventArgs e)
    {
        if (e.ChangeType == WatcherChangeTypes.Created)
        {
            // ...
            _serviceA.Run();
            // ...
        }
    }
    //...
}

All done. Everything is hooked up. However, when we run the program, we will get an error immediately. The error message is similar to the following.

System.AggregateException: Some services are not able to be constructed (Error while validating the service descriptor 'ServiceType: Microsoft.Extensions.Hosting.IHostedService Lifetime: Singleton ImplementationType: Demo.Worker': Cannot consume scoped service 'Demo.Services.IServiceA' from singleton 'Microsoft.Extensions.Hosting.IHostedService'.)

The key part of the error message is “cannot consume scoped service from singleton”. This error is due to the Worker service has a singleton lifetime, while the ServiceA has a scoped lifetime, which would be garbage collected before the Worker service. Thus they cause a violation in DI container.

In order to make it work, we can borrow the IServiceProvider to create a scope and resolve the scoped service. The following code snippet shows an example.

Worker.cs

public class Worker : BackgroundService
{
    // ...
    private readonly IServiceProvider _services;

    public Worker(ILogger<Worker> logger, IOptions<AppSettings> settings, IServiceProvider services)
    {
        // ...
        _services = services;
    }
    // ...
    protected void Input_OnChanged(object source, FileSystemEventArgs e)
    {
        if (e.ChangeType == WatcherChangeTypes.Created)
        {
            // ...
            using (var scope = _services.CreateScope())
            {
                var serviceA = scope.ServiceProvider.GetRequiredService<IServiceA>();
                serviceA.Run();
            }
            // ...
        }
    }
    //...
}

In this way, the DI container is able to resolve all dependencies. Problem solved!

Now, we are ready to deploy our application to a Windows Service.

Windows Service management

To achieve best performance, we first need to build our application in the Release mode.

If you have PowerShell 6.2+, then you can use a series of commands to install, start/stop, remove Windows Services. These commands include New-Service, Start-Service, Get-Service, Stop-Service, and Remove-Service.

BTW: Make sure you have Admin privilege to do these operations.

Here, we will take the old fashion approach to manage Windows Services using sc.exe in CMD. Note: in CMD environment only. The sc commands might not work in PowerShell environment.

The commands to create, start/stop, and delete a Windows Service are shown in the following code snippet.

cmd.bat

:: Create a Windows Service
sc create DemoService DisplayName="Demo Service" binPath="C:\full\path\to\Demo.exe"

:: Start a Windows Service
sc start DemoService

:: Stop a Windows Service
sc stop DemoService

:: Delete a Windows Service
sc delete DemoService

It is worth mentioning that the binPath needs to be the full path of the exe file of the application. All the commands should be easy to use.

Looks like we have covered all the stuff as planed. Hope you have learned something new, and I bet you are able to get started to create a Windows Service app in .NET Core 3 now.

If you need reference, you can find the full project in this GitHub repository.

Thanks for reading.