Sync Root Hashes
with Negentropy
Sync File Chunks
using LeafSync
Customizable
Nostr Relay
File Hosting
and Syncing
Deduplicate all hosted files with their hashes: never store the same file twice.
Store files from users as Blossom Blobs or Scionic Merkle Trees. Users attach files to a post by referencing its hash.
History of Merkle Trees
Hashed tree structures have a long open-source legacy. From the invention of Merkle Trees by Ralph Merkle in the 80s, later adopted by bitcoin, to Git’s use of hashed DAGs and IPFS's implementation of Merkle DAGs for file storage. However, Classic Merkle Trees cannot contain a folder of files. Merkle DAGs were developed as a solution to incorporate a directory of files into a similar Merklized structure.
However, Merkle DAGs require users to download the hash of every other file inside the folder its stored in. This becomes inefficient when dealing with folders that contain large amounts of files, or very large single files. You must download the hashes to all of the other chunks to retrieve a single file. This requirement can lead to large Merkle DAG branches that Scionic Merkle Trees avoid.
Scionic Merkle Trees exhibit logarithmic growth. This means that as the number of files in a folder increases, the size of each Scionic Merkle branch decreases. This makes Scionic Merkle Trees an efficient structure for managing large datasets, as the branches become exponentially smaller with the increasing number of files in the folder.
Best of Both Worlds: Folder Storage & Small Branches
Scionic Merkle Tree are a new type of Merkle Tree/Merkle DAG structure for file chunking. Scionic Merkle Trees have small branches like Classic Merkle trees, the file/folder storage ability of Merkle DAGs, and numbered leaves so relays can request a range of missing leaf numbers in order to sync succinctly. Merging the advantages of Classic Merkle trees and Merkle DAGs gives H.O.R.N.E.T Storage the best of both worlds: file storage and small branches. This will unlock a new tier of speed for a variety of decentralized apps that require large file directories.
In plant grafting, the scion is the upper part of the plant, chosen for its desirable fruits or flowers. It's grafted onto another plant's base to grow together. In a similar vein, the Scionic Merkle Tree was born from grafting together Merkle DAGs and Classic Merkle Trees. This process emphasizes why we use the term "scion" for the Scionic Merkle Trees. The name symbolizes the digital grafting of these two similar data structures, combining their strengths into one piece.
Airlock Uses Scionic Merkle Trees for GitNestr Permissions
Include Phase D here on launch day.
Scionic Merkle Tree
Customizable
Nostr Relay
File Hosting
and Syncing
Deduplicate all hosted files with their hashes: never store the same file twice.
Store files from users as Blossom Blobs or Scionic Merkle Trees. Users attach files to a post by referencing its hash.
Scionic Merkle Branch
Scionic Merkle Tree
First, the Scionic Merkle Root of each file is synced with Negentropy's RBSR. The Merkle root of each file is inspected so the total number of leaves/chunks for that file is revealed. After, H.O.R.N.E.T Storage checks if any of those file chunks/leaf numbers are currently saved. Finally, LeafSync is initiated to request the missing file chunks by listing a range of leaf numbers, e.g. "merkle_root:1-50" that represents a request for the missing file chunks.
