Caching shared application data in dot net

[d000hg]

So the built-in caching in .Net doesn't do such things SY? I haven't used it other than as a way to make data automatically expire after a given time period so I'd be interested to hear what you find out.

[suityou01]

I for one think this is an inneresting technical question.

Is the data going to be shared across different processes and or servers or loaded into one process?

How much memory will these large objects occupy?

In terms of stale data, one pattern is a write through cache. All access to the DB / persistence store is through the write through cache. This way the cache always knows when data has been invalidated. You can even have a lazy writer process persist the data from the cache in the background (write behind).

There are many open source and cloud solutions. Memcached, AWS Elasticache, Azure Redis Cache for example.

Thanks DP

The data would be shared across processes and servers. The data is really only likely to be a few 100,000 rows of normal text on some fairly wide tables so not really as bad as I was making out with the blobs, but I was exaggerating to see if some super duper frameworks would come to the fore.

This isn't cloud but on premise.

The write through cache is exactly the kind of thing I was thinking of, but the first rule is don't cut code if there is something already available.

I will look at the Open Source solutions you mention.

[BigRed]

... but it works on his 3 rows in Development

[DimPrawn]

.net has a built in memorycache, key, value. You could hash the SQL criteria and use that as the key.

You can, that's why I asked if the cache needs to be distributed and maintained across multiple processes and machines and how large the data is.

If it's 4TB of data to cache, good luck holding that in one .Net process.

[woohoo]

.net has a built in memorycache, key, value. You could hash the SQL criteria and use that as the key.

In the past when changing config data through an admin site, I've called a web service to invalidate cache which would force the next query to fetch the data from the db.

[DimPrawn]

Hi,

Any frameworks anyone can suggest for this?

Say my app runs this

SELECT BigFatMahoosiveBlob FROM BigFatTable WHERE Predicate = xyz

After it's done the heavy lifting over the wire from the DB, I want to cache this on my Application server (In memory) and then when a query with exactly the same predicate comes along, my app can go "tadaaaaa" without having to do a fresh DB fetch.

Usual expiration stuff etc.

So, what does the panel recommend?

For bonus points, is there anything where the DB can ping back a message to say you're data is stale now?

I for one think this is an inneresting technical question.

Is the data going to be shared across different processes and or servers or loaded into one process?

How much memory will these large objects occupy?

In terms of stale data, one pattern is a write through cache. All access to the DB / persistence store is through the write through cache. This way the cache always knows when data has been invalidated. You can even have a lazy writer process persist the data from the cache in the background (write behind).

There are many open source and cloud solutions. Memcached, AWS Elasticache, Azure Redis Cache for example.

[TheFaQQer]

Put you requirements down.

Optimistic

[MrMarkyMark]

Hi,

Any frameworks anyone can suggest for this?

Say my app runs this

SELECT BigFatMahoosiveBlob FROM BigFatTable WHERE Predicate = xyz

After it's done the heavy lifting over the wire from the DB, I want to cache this on my Application server (In memory) and then when a query with exactly the same predicate comes along, my app can go "tadaaaaa" without having to do a fresh DB fetch.

Usual expiration stuff etc.

So, what does the panel recommend?

For bonus points, is there anything where the DB can ping back a message to say you're data is stale now?

Know your limits

Put you requirements down.

LM and I will be back in a few days with timescales, resource requirements and a quote

[Jaws]

I'd have to search MSDN because it's been so long but I think you're looking for the SqlDependency class assuming you're using SQL server which I believe will provide some mechanism of cache invalidation when data becomes stale. Perhaps there is something similar available for different database servers.

Have you relocated to Bangalore?

Man up, do your job or hand your pass in you fraud.

[LondonManc]

Have you relocated to Bangalore?

Man up, do your job or hand your pass in you fraud.

[suityou01]

Hi,

Any frameworks anyone can suggest for this?

Say my app runs this

SELECT BigFatMahoosiveBlob FROM BigFatTable WHERE Predicate = xyz

After it's done the heavy lifting over the wire from the DB, I want to cache this on my Application server (In memory) and then when a query with exactly the same predicate comes along, my app can go "tadaaaaa" without having to do a fresh DB fetch.

Usual expiration stuff etc.

So, what does the panel recommend?

For bonus points, is there anything where the DB can ping back a message to say you're data is stale now?