My own pitfalls with .NET runtime cache

I was working on a ASP.NET-project the other day where we use a runtime cache (aka. application cache) that lives for the duration of the application lifetime. We use this to store some frequently used data and we update the cache when something changes.

The cache implementation is not state of the art and I figured I’ll share some learnings and pitfalls that I’ve fallen into over the years.

Mutable objects in the runtime cache

First of all: An mutable object, in contrast to an immutable object, is an object that can change it's state (aka properties on the object can change value without having to create a new object). Since an immutable object can't alter its state, we need to create a new instance of the object if we need to change any values. In .NET a standard class with get/set properties is mutable while DateTime, TimeSpan, and many others are immutable.

Years ago one of the biggest gotcha for me with using the MemoryCache in .NET is that it will actually store objects. Not serialized objects but real objects in memory and only pass the reference to any consumer.

This is of course great for performance, but it also means that one has to be very careful about how these objects are used. We could deep clone the object when fetching it from the cache to avoid many of the issues I’m going to point out here but in our case, we used the “vanilla memory cache” in .NET.

Since the objects are mutable, we can easily change the state, ie. change a property or add an item to a list – we just need to remember that the next time this object is accessed the new values will be there, and the old values are gone.

Updating values

Have a look at this code sample:

public class SomeService {

    public bool UpdateCustomer(CustomerViewModel vm)
    {
        // Getting the value from the CustomerService, which is wrapped in a 
        // caching-decorator that uses the .NET MemoryCache.
        var customer = _customerService.GetCustomer(vm.CustomerId);

        customer.Name = vm.Name;
        customer.City = vm.City;
        customer.MaxOrderAmount = vm.MaxOrderAmount;

        var saveResult = _customerService.Save(customer);

        if(saveResult.HasValidationErrors)
            return ValidationError(saveResult);

        return Success();

    }
}

As you can see, we’re applying the changes from the view model into the Customer model and then saving it with the CustomerService which will validate the Customer before saving it. Let’s say that there is a validation error, the service will set HasValidationErrors to true and we’ll return the issues to the view.

BUT! This code contains a nasty bug. Since the GetCustomer()-method returns an object from the cache, the changes we make to the object (setting the values from the view model) will be persisted in the cache no matter if the validation is successful or not. This is all very logical and makes sense but it’s a big “gotcha” in terms of how caching works.

Another thing that has happened to me over the years: I was reading an object from the cache that had related entities (think customers with a List<Order>). I wanted to pass a Customer together with only paid orders to another service so I modified the order-property on the Customer like so:

customer.Orders = customer.Orders.Where(x=>x.Paid == true).ToList();

This felt great and the service that I called could use the customer-object from the cache. The only problem is that the underlying collection of orders is modified and the next time I read the Customer from the cache only the paid orders will be in the collection.

Threading and runtime cache

Most of the time the in-memory runtime cache would be shared inside the application, since I’m mostly doing ASP.NET this would be all threads used by the webserver to process requests.

Here we need to keep in mind that while one thread might be reading the cache, getting a reference to an object to read it – another thread might be in the processing of updating values on the same object, it might even be in the middle of this processes and depending on implementation the object might be in an invalid state (one property has been updated but not the other) causing errors on the read-side since the values do not make sense.

Solutions?

Going forward I can see a couple of things that would make it harder to “do it wrong”.

• Always use un-cached business objects when modifying state. (ie. the method above should not read from the cache). This way we can safely apply changes to mutable objects and validate like in the sample above.
• Cache a “read-only”-representation of the underlying business object. This representation could be a CustomerReadOnly-class with private setters for all the properties. This way the consuming code can’t change the state by mistake.
• Use C# 9 record types, they are immutable so it's impossible to change the state of the cached object. If changes are needed on "the read side" a new instance of the record would have to be created - which will not impact the cached object. This way, any "implicit" changes to the cache are impossible.

There is a lot more to this subject but I figured I’ll post this as a starting point.