Scheduling Background Jobs With Quartz.NET

If you're building a scalable application, it's a common requirement to offload some work in your application to a background job.

A few examples of that are:

Publishing email notifications
Generating reports
Updating a cache
Image processing

How can you create a recurring background job in .NET?

Quartz.NET is a full-featured, open source job scheduling system that can be used from smallest apps to large scale enterprise systems.

There are three concepts you need to understand in Quartz.NET:

Job - the actual background task you want to run
Trigger - the trigger controlling when a job runs
Scheduler - responsible for coordinating jobs and triggers

Let's see how we can use Quartz.NET to create and schedule background jobs.

Adding The Quartz.NET Hosted Service

The first thing we need to do is install the Quartz.NET NuGet package. There are a few to pick from, but we're going to install the Quartz.Extensions.Hosting library:

Install-Package Quartz.Extensions.Hosting

The reason we're using this library is because it integrates nicely with .NET using an IHostedService instance.

To get the Quartz.NET hosted service up and running, we need two things:

Add the required services with the DI container
Add the hosted service

services.AddQuartz(configure =>
{
    configure.UseMicrosoftDependencyInjectionJobFactory();
});

services.AddQuartzHostedService(options =>
{
    options.WaitForJobsToComplete = true;
});

Quartz.NET will create jobs by fetching them from the DI container. This also means you can use scoped services in your jobs, not just singleton or transient services.

Setting the WaitForJobsToComplete option to true will ensure that Quartz.NET waits for the jobs to complete gracefully before exiting.

Creating Background Jobs With `IJob`

To crate a background job with Quartz.NET you need to implement the IJob interface.

It only exposes a single method - Execute - where you can place the code for your background job.

A few things worth noting here:

We're using DI to inject the ApplicationDbContext and IPublisher services
The job is decorated with DisallowConcurrentExecution to prevent running the same job concurrently

[DisallowConcurrentExecution]
public class ProcessOutboxMessagesJob : IJob
{
    private readonly ApplicationDbContext _dbContext;
    private readonly IPublisher _publisher;

    public ProcessOutboxMessagesJob(
        ApplicationDbContext dbContext,
        IPublisher publisher)
    {
        _dbContext = dbContext;
        _publisher = publisher;
    }

    public async Task Execute(IJobExecutionContext context)
    {
        List<OutboxMessage> messages = await _dbContext
            .Set<OutboxMessage>()
            .Where(m => m.ProcessedOnUtc == null)
            .Take(20)
            .ToListAsync(context.CancellationToken);

        foreach (OutboxMessage outboxMessage in messages)
        {
            IDomainEvent? domainEvent = JsonConvert
                .DeserializeObject<IDomainEvent>(
                    outboxMessage.Content,
                    new JsonSerializerSettings
                    {
                        TypeNameHandling = TypeNameHandling.All
                    });

            if (domainEvent is null)
            {
                continue;
            }

            await _publisher.Publish(domainEvent, context.CancellationToken);

            outboxMessage.ProcessedOnUtc = DateTime.UtcNow;

            await _dbContext.SaveChangesAsync();
        }
    }
}

Now that the background job is ready, we need to register it with the DI container and add a trigger that will run the job.

Configuring the Job

I mentioned at the start that there are three key concepts in Quartz.NET:

Job
Trigger
Scheduler

We already implemented the ProcessOutboxMessagesJob background job in the previous section.

The Quartz.NET library will take care of the scheduler.

And this leaves us with configuring the trigger for our ProcessOutboxMessagesJob.

services.AddQuartz(configure =>
{
    var jobKey = new JobKey(nameof(ProcessOutboxMessagesJob));

    configure
        .AddJob<ProcessOutboxMessagesJob>(jobKey)
        .AddTrigger(
            trigger => trigger.ForJob(jobKey).WithSimpleSchedule(
                schedule => schedule.WithIntervalInSeconds(10).RepeatForever()));

    configure.UseMicrosoftDependencyInjectionJobFactory();
});

We need to uniquely identify our background job with a JobKey. I like to keep it simple and use the job name.

Calling AddJob will register the ProcessOutboxMessagesJob with DI and also with Quartz.

After that we configure a trigger for this job by calling AddTrigger. You need to associate the job with the trigger by calling ForJob, and then you configure the schedule for the background job. In this example, I'm scheduling the job to run every ten seconds and repeat forever while the hosted service is running.

Quartz also has support for configuring triggers using cron expressions.

Job Persistence

By default, Quartz configures all jobs using the RAMJobStore which is the most performant because it keeps all of its data in RAM. However, this also means it's volatile and you can lose all scheduling information when your application stops or crashes.

It could be useful to have a persistent job store in some scenarios and there's a built in AdoJobStore which works with SQL databases. You need to create a set of database tables for Quartz.NET to use.

You can learn more about this in the job stores documentation.

Takeaway

Quartz.NET makes running background jobs in .NET easy, and you can use all the power of DI in your background jobs. It's also flexible for various scheduling requirements with configuration via code or using cron expressions.

There's some room for improvement to make scheduling jobs easier and reduce boilerplate:

Add an extension method to simplify configuring jobs with a simple schedule
Add an extension method to simplify configuring jobs with a cron schedule from application settings

If you want to see a tutorial on using Quartz.NET, I made an in-depth video about using Quartz for processing Outbox messages.

That's all for this week.

See you next Saturday.

P.S. Whenever you’re ready, there are 2 ways I can help you:

Pragmatic Clean Architecture: This comprehensive course will teach you the system I use to ship production-ready applications using Clean Architecture. Learn how to apply the best practices of modern software architecture. Join 950+ students here.
Patreon Community: Think like a senior software engineer with access to the source code I use in my YouTube videos and exclusive discounts for my courses. Join 820+ engineers here.