Systems Papers

Share this post

Systems Papers - Shard Manager: A Generic Shard Management Framework for Geo-distributed Applications

newsletter.micahlerner.com

Systems Papers - Shard Manager: A Generic Shard Management Framework for Geo-distributed Applications

Micah Lerner
Jan 9, 2022
Share this post

Systems Papers - Shard Manager: A Generic Shard Management Framework for Geo-distributed Applications

newsletter.micahlerner.com

Shard Manager: A Generic Shard Management Framework for Geo-distributed Applications

Hello new and old subscribers!

This week’s paper, Shard Manager: A Generic Shard Management Framework for Geo-distributed Applications, describes a system for running sharded applications at scale within Facebook - link to the paper review here.

Application sharding assigns subsets of requests to instances of an application, allowing tasks to specialize - a group of tasks specialized for a subset of requests is called a shard.

This approach is particularly useful when tasks fetch state or other metadata. As an example, a speech recognition application may load machine learning models to process languages. Assigning requests for different languages to different shards means that an application’s tasks don’t need to download every model (which would be time and bandwidth-intensive).

The paper review is best read on my blog.

As always, feel free to reach out with comments or suggestions by responding to this email.

Until next time!

Micah

Share this post

Systems Papers - Shard Manager: A Generic Shard Management Framework for Geo-distributed Applications

newsletter.micahlerner.com
Comments
TopNew

No posts

Ready for more?

© 2023 Micah Lerner
Privacy ∙ Terms ∙ Collection notice
Start WritingGet the app
Substack is the home for great writing