Today most blockchain dApps are very simple and require very little state to be stored. However, as this ecosystem matures, there will be more complex dApps that get built (think a decentralized youtube). Storing data created by these dApps (comments, user uploads etc.) on-chain is very expensive. And using a centralized system like Amazon S3 defeats the purpose of decentralized computing. So there is a need for cheap, decentralized storage that is off-chain.
One of the key technologies that is employed to solve this problem is InterPlanetary File System or IPFS. It is a peer to peer (p2p) file sharing system that aims to change the way data is shared around the globe. Similar to Torrent clients, IPFS allows individuals to share data with each other. IPFS allows any person with unused storage on their laptops or desktops to provide this storage to the network, thus becoming a participant in the IPFS ecosystem. This decentralized framework to share data reduces the power central authorities have to censor data.
Back in 2017 when the Turkish government banned Wikipedia, the IPFS team aided the country’s citizens by uploading a copy of the Turkish version of Wikipedia on it’s network. As IPFS file objects are not stored on any centralised servers, the Turkish government could not take down the uploaded files. Remarkable, right?
Let’s dive into this technology a bit more.
Imagine you are in a classroom and the professor shares a link to a 200 mb video with the entire class. The current set of internet protocols essentially require 200 mb of data to be downloaded by each student in the classroom. This means that each individual student uses 200mb of bandwidth to access this file from a centralised server. Now, keep in mind, this centralized server could be located halfway across the world and how fast data moves is limited by the speed of light.
Now imagine if you and your classmates could end up downloading all the content directly from your professor’s laptop.
This in a nutshell is what IPFS enables. It proposes an ecosystem where the relevant data would be fetched from other people participating in the network — often located a lot closer to you than the original server hosting the content. This would help you download the data faster and could potentially save your internet bandwidth assuming the data is transferred on an internal network.
Like blockchain, IPFS also acts as an immutable data storage platform. Every file uploaded in this network gets cryptographically secured and gets a unique hash assigned to it. Similar to human fingerprints, these hashes are distinctive and irreplicable thereby preventing duplication of identical files.
Unlike records on a blockchain, IPFS files can be updated. The protocol allows users to upload newer versions their files. The very first file upload creates a “commit object” and every subsequent update to the file creates another “commit object” with a link to the previous “commit object”. Consequently, earlier versions of the file can be easily retrieved. For you developers reading this article, this form of version control was inspired by git.
Files on the IPFS network are stored in what are called “IPFS objects”. These objects can store upto 256 kilobytes worth of data. Along with the contents of a file, the objects can also contain links to other IPFS objects. So a simple text file can fit into one single object while an mp4 file would be split up across several objects of not more than 256kb in size. In case a file is split into separate IPFS objects, the system also creates another IPFS object with a schema outlining how the IPFS objects are linked together.
Even with all the work put into building this system, as with most new technologies, IPFS is not without its limitations.
In our opinion, the biggest problem that IPFS faces is keeping files available. Every node or participant on the network keeps a record of the files it has downloaded and shares these files with other people when necessary. But what if, at the time a file is required by a network participant, all nodes holding copies of that file are offline? This could happen as there is no incentive built into IPFS for a node to host a file on the IPFS network.
A possible solution to this problem has been proposed by the same team that built IPFS — Filecoin. It is a blockchain build on top of IPFS that aims to create a decentralised marketplace for storage with network participants being incentivised to behave well.
Filecoin is yet to be launched but we at the Crypto University are super excited about its underlying technology. Decentralized apps require a decentralized storage platform and IPFS could provide this storage layer.
Fun Fact: The reason it was named Interplanetary File system is because, when humans become a interplanetary species, it would be sensible to fetch the data from the closest user rather than from planets light years away :)