• Visitors can check out the Forum FAQ by clicking this link. You have to register before you can post: click the REGISTER link above to proceed. To start viewing messages, select the forum that you want to visit from the selection below. View our Forum Privacy Policy.
  • Want to receive the latest contracting news and advice straight to your inbox? Sign up to the ContractorUK newsletter here. Every sign up will also be entered into a draw to WIN £100 Amazon vouchers!

Caching shared application data in dot net

Collapse
X
  •  
  • Filter
  • Time
  • Show
Clear All
new posts

    Caching shared application data in dot net

    Hi,

    Any frameworks anyone can suggest for this?

    Say my app runs this

    Code:
    SELECT BigFatMahoosiveBlob FROM BigFatTable WHERE Predicate = xyz
    After it's done the heavy lifting over the wire from the DB, I want to cache this on my Application server (In memory) and then when a query with exactly the same predicate comes along, my app can go "tadaaaaa" without having to do a fresh DB fetch.

    Usual expiration stuff etc.

    So, what does the panel recommend?

    For bonus points, is there anything where the DB can ping back a message to say you're data is stale now?
    Knock first as I might be balancing my chakras.

    #2
    Have you relocated to Bangalore?

    Man up, do your job or hand your pass in you fraud.
    The greatest trick the devil ever pulled was convincing the world that he didn't exist

    Comment


      #3
      I'd have to search MSDN because it's been so long but I think you're looking for the SqlDependency class assuming you're using SQL server which I believe will provide some mechanism of cache invalidation when data becomes stale. Perhaps there is something similar available for different database servers.

      Originally posted by LondonManc View Post
      Have you relocated to Bangalore?

      Man up, do your job or hand your pass in you fraud.

      Comment


        #4
        Originally posted by suityou01 View Post
        Hi,

        Any frameworks anyone can suggest for this?

        Say my app runs this

        Code:
        SELECT BigFatMahoosiveBlob FROM BigFatTable WHERE Predicate = xyz
        After it's done the heavy lifting over the wire from the DB, I want to cache this on my Application server (In memory) and then when a query with exactly the same predicate comes along, my app can go "tadaaaaa" without having to do a fresh DB fetch.

        Usual expiration stuff etc.

        So, what does the panel recommend?

        For bonus points, is there anything where the DB can ping back a message to say you're data is stale now?
        Know your limits

        Put you requirements down.

        LM and I will be back in a few days with timescales, resource requirements and a quote
        The Chunt of Chunts.

        Comment


          #5
          Originally posted by MrMarkyMark View Post
          Put you requirements down.
          Optimistic
          Best Forum Advisor 2014
          Work in the public sector? You can read my FAQ here
          Click here to get 15% off your first year's IPSE membership

          Comment


            #6
            Originally posted by suityou01 View Post
            Hi,

            Any frameworks anyone can suggest for this?

            Say my app runs this

            Code:
            SELECT BigFatMahoosiveBlob FROM BigFatTable WHERE Predicate = xyz
            After it's done the heavy lifting over the wire from the DB, I want to cache this on my Application server (In memory) and then when a query with exactly the same predicate comes along, my app can go "tadaaaaa" without having to do a fresh DB fetch.

            Usual expiration stuff etc.

            So, what does the panel recommend?

            For bonus points, is there anything where the DB can ping back a message to say you're data is stale now?
            I for one think this is an inneresting technical question.

            Is the data going to be shared across different processes and or servers or loaded into one process?

            How much memory will these large objects occupy?

            In terms of stale data, one pattern is a write through cache. All access to the DB / persistence store is through the write through cache. This way the cache always knows when data has been invalidated. You can even have a lazy writer process persist the data from the cache in the background (write behind).

            There are many open source and cloud solutions. Memcached, AWS Elasticache, Azure Redis Cache for example.

            Comment


              #7
              .net has a built in memorycache, key, value. You could hash the SQL criteria and use that as the key.

              In the past when changing config data through an admin site, I've called a web service to invalidate cache which would force the next query to fetch the data from the db.
              Last edited by woohoo; 28 March 2017, 21:11.

              Comment


                #8
                Originally posted by woohoo View Post
                .net has a built in memorycache, key, value. You could hash the SQL criteria and use that as the key.
                You can, that's why I asked if the cache needs to be distributed and maintained across multiple processes and machines and how large the data is.

                If it's 4TB of data to cache, good luck holding that in one .Net process.

                Comment


                  #9
                  ... but it works on his 3 rows in Development

                  Comment


                    #10
                    Originally posted by DimPrawn View Post
                    I for one think this is an inneresting technical question.

                    Is the data going to be shared across different processes and or servers or loaded into one process?

                    How much memory will these large objects occupy?

                    In terms of stale data, one pattern is a write through cache. All access to the DB / persistence store is through the write through cache. This way the cache always knows when data has been invalidated. You can even have a lazy writer process persist the data from the cache in the background (write behind).

                    There are many open source and cloud solutions. Memcached, AWS Elasticache, Azure Redis Cache for example.

                    Thanks DP

                    The data would be shared across processes and servers. The data is really only likely to be a few 100,000 rows of normal text on some fairly wide tables so not really as bad as I was making out with the blobs, but I was exaggerating to see if some super duper frameworks would come to the fore.

                    This isn't cloud but on premise.

                    The write through cache is exactly the kind of thing I was thinking of, but the first rule is don't cut code if there is something already available.

                    I will look at the Open Source solutions you mention.
                    Knock first as I might be balancing my chakras.

                    Comment

                    Working...
                    X