Hey hackernews! I'd like to share a hobby and passion project I've been working on for almost a year now. It's an open-source storage engine (underlying component of a database(s)) similar to that of LevelDB/RocksDB but written entirely in C.

The storage engine is called TidesDB.

TidesDB is written entirely in C. It's designed as a fast, transactional key-value storage engine built on a log-structured merge-tree (LSM-tree) architecture.

My journey with TidesDB began nearly 2 years ago whilst I was experimenting with various data structures and databases in Go. When I encountered the log-structured merge tree concept, I was initially overwhelmed by its complexity after reviewing other implementations.

However, after studying the original paper, I realized I could potentially simplify the design by focusing on just a 2-level approach(memory level and a disk level). This was challenging at first, and I scrapped many ideas along the way, but over time I discovered how powerful this design could potentially be.

The beauty of this design/architecture is its extensibility. Since the disk level contains many SSTables (Sorted String Tables), we can efficiently pair and merge them in various ways - whether in parallel for speed or incrementally to minimize resource impact.

What began as a challenging learning process has I believed evolved into a unique engine design and library.

You can check out TidesDB code here: https://github.com/tidesdb/tidesdb

Website here (Architecture-design, documentation and more): https://tidesdb.com

There are a variety of FFI libraries in the works for C++, GO, Python, Lua and more!

Currently TidesDB is nearing its first major release, we are still in beta development.

I'd love to get your thoughts, questions and all :)