OpenACID Blog

Distributed systems research notes.

Recent posts

The Pitfalls of Raft Membership Change

Raft’s single-server membership change carries two problems: its middle state can turn one datacenter into a single point of failure, and the original algorithm could lose a committed entry. Let us look at membership change through quorum sets, and see why joint consensus is the answer.

Build a Distributed Timestamp Oracle with EzRaft

timestamp_oracle is a 200-line timestamp service on EzRaft. A background task reserves ranges through Raft; next_timestamp allocates from memory. Leader leases and term-scoped caches keep the sequence strictly increasing across failovers, with no Raft write on the request path.

EzRaft: Build a Distributed KV Store in 100 Lines

EzRaft builds on OpenRaft and hides the storage, networking, and service layers. Define just four things—Request, Response, apply, and read—and you have a fault-tolerant, three-node distributed KV store.

Histogram Done Right: 2KB Memory, 0.2% Error

A lightweight histogram that tracks latency distributions in 2KB of memory with sub-0.2% error. Uses a float-like encoding for O(1) bucket indexing and trapezoid interpolation for accurate percentile estimation — no floating-point math needed.

Raft Node Rejoin Bug

Analyzes a replication session isolation bug in Raft implementations. When a node rejoins the cluster within the same term, delayed AppendEntries responses can corrupt progress tracking, causing infinite retry loops. While data safety remains intact, it creates operational issues like resource exhaustion. Uses raft-rs as a case study to examine trigger conditions and solutions.

Raft IO Execution Order (Revised)

I got it wrong in my previous article. The IO ordering bug in Raft isn’t about the protocol design—it’s about the subtle trap that emerges when implementations split state into in-memory and persisted state. Here’s what actually happens.

The Hidden Danger in Raft: Why IO Ordering Matters

Writing logs before persisting term in Raft can silently destroy committed data. Here’s why production systems like TiKV and HashiCorp Raft carefully control IO order—and three battle-tested solutions.

Paxos 中 Ballot number 的重用机制

本文探讨 Paxos 协议中 Ballot number 的重用机制,分析何时可重复使用,以及为何只能使用系统中已存在的值而非提议新值

Multi-Master-Paxos: 3

小孩子才选master, 成年人只用multi-master

Raft 成员变更上踩的坑

Raft 在单步成员变更的设计上存在缺陷, 一定要抛弃单步变更, 使用joint consensus

200行代码实现基于paxos的kv存储

用200行代码实现一个基于paxos的kv存储, 以最简洁的形式展示paxos如何运行, 作为 paxos的直观解释 这篇教程中的代码示例部分

后分布式时代: 多数派读写的’少数派’实现

通过减少多数派读写中参与者的数量, 可以实现分布式系统某些特定场景的优化, 以及介绍这些优化对系统可用性产生的影响, 根据什么标准来选择和调整这些参数.

Erasure-Code-擦除码-3-极限篇

没有人比我更懂EC(特辣脯口气): 从指令集和算法两个方向对擦除码实现深度优化, 硬核但不烧脑

Erasure-Code-擦除码-2-实现篇

本文介绍了如何在计算机有限的数字精度内实现高效的擦除码算法,并通过引入伽罗华域及模运算,详细阐述了从GF(2)扩展到GF(2⁸)并实现EC编码、解码的过程,最后还讨论了不同矩阵的适用性

Erasure-Code-擦除码-1-原理篇

擦除码(EC)通过增加校验块来保证分布式存储的可靠性,允许一定数量的数据块丢失也能恢复。文章以RAID-5、RAID-6为例,解释了k+1、k+2校验策略,并推广到更通用的k+m策略,将其类比为用k个系数定义曲线,并通过曲线上点坐标实现数据冗余

Art of Pull Requests(翻译)

保持 PR 小而快,方便快速审查和合并。 提供清晰的上下文信息,方便审阅者理解代码变更。 积极友好的审查方式,专注于代码逻辑和架构。 团队成员之间彼此信任,并清晰描述代码变更意图。

掐指算算: 你的CDN多花了几百万?

本文分析了内容分发网络 (CDN) 的成本构成,以及 Zipf 定律对成本的影响。CDN 利用边缘机房缓存热门内容,降低回源成本。文章通过量化业务参数、拟合 Zipf 曲线、确定成本数据,找到了使存储和带宽成本总和最小的最佳边缘容量配置。最后,文章展望了 CDN 和源站未来更紧密的协作模式,以实现更高效的内容分发

互联网中对象访问频率的91分布

本文介绍了Zipf定律,该定律描述了互联网内容访问频率分布规律:少数热门内容占据了大部分流量。文章通过数学推导,得到Zipf模型公式,并解释了如何用多项式回归拟合访问日志数据,确定公式参数。最后,文章列举了该定律在缓存配置、日志分析等场景的应用

SlimTrie: 单机百亿文件的极致索引-实现篇

SlimTrie的代码级实现详解:通过Trie裁剪去除单分支节点将空间从O(n*k)降至O(n),再用compacted array压缩存储,最终每条索引仅需约6字节(加4字节value共10字节)。实测内存开销约为B-Tree的13%,查询速度约为其2.6倍。

SlimTrie: 单机百亿文件的极致索引-设计篇

单机100TB存储如何索引百亿小文件?经典的hash map和B-Tree索引的内存开销为O(k*n),无法全部放入内存。SlimTrie基于Trie裁剪和压缩,将空间复杂度降至O(n),实现每条索引仅约10字节,在1GB内存中即可建立100TB数据量的索引。