Deep Dive Inside a Distributed Cache Engine

Core components while designing a distributed, scalable and fault-tolerant cache system.

Caching data helps to improve application performance. Failing the caching service will put an extreme load on the database and result in poor performance, in the worst case, can crash the service. While we design a caching service, we should consider low latency at the minimum cost possible. Depending on the scenario and application requirements, we should choose the appropriate and affordable caching mechanism. A caching mechanism should offer,

Read Through: Application first go to the cache-store to fetch the data

If data exist in the cache-store it returns the data to the application

When data does not exist in the cache-store, the cache itself fetch the data from the database

Write Through: Storing data to Database can be handled by either the Cache or the Application.

When the data persist is handled by Cache, the application first writes data to the Cache store and then Cache store write data to Database. It can be time-consuming, because, we need validation that both cache and database have persisted the data synchronously. If the cache goes down before the data is persisted in the database, the data can be lost.

Cache Aside: With this approach, data will be persisted in cache and database both handled by the application. In this case, failure of the caching mechanism will not lose the data.

Write Back / Behind Cache: Another hybrid architecture can be store data in the cache initially and after a certain period/threshold we will persist all these data in the database as a bulk insert.

Read / Refresh Cache: Data is cached before the user looked for it. Use some prediction engine or machine learning model to decide which data should be loaded. If we know, the user will look for the followers feed, we will load it the moment user logged in and show the data when asked.

Depending on the application requirements, we have to decide how close the cache and application server will be.

It could be,

If we put the cache along with each application server, it will be the faster response. But considering if a server fails, the cache will also fail. Also, there will be no sync of cache data between servers.

On the other hand, if we use a global database, even though an application fails, the cache data will still be available. In this case, although it is a comparatively slow response, still more accurate and we can scale the caching mechanism independently.

Cache memory is limited and we need a defined policy on how we make cache data invalid.

When it comes to designing a distributed caching mechanism, we have to consider,

Fault-tolerant: We can use multiple servers and replicate each server’s data in other servers. In this case, even if one server is lost, we can use the replicated data. To manage this data replication, we can do one of the following approaches,

System Topology: All their master/child servers’ data, their read-write operations, a proxy server to handle the Client SDK will be handled by the topology manager.

There’s a lot when it comes to designing a distributed caching mechanism like Redis. These are the core components when we design such a large-scale system. Feel free to reach out for any queries.

Related posts:

The United States of America holds 716 in 100,000 people of the nations population in prison or 22.2% of the world’s population. The majority of those inmates are returning drug offenders such as…

Cryptomarket had a dramatic start with the unprecedented bulls from 2017 coming to rest, losing over 60% of total market cap value in just a month of January. Further, 2018 was even more pessimistic…

Looking for a juicer for greens? Take a closer look at auger models: they are easy to use and great for processing not only leaves and sprouts of plants, but also the seeds, and even nuts. What…