A Complete Guide to IPLD: Its Significance, How it Works and Practical Examples This post is a continuation(part 2) in a new “ ” series which will help anybody to understand the underlying concepts of IPFS. If you want an overview of what is IPFS and how it works, then you should check out the first part too 😊 Understanding IPFS in Depth Understanding IPFS in Depth(1/6): A Beginner to Advanced Guide In our first part of the series we briefly discussed IPLD. We saw that IPLD deals with “defining the data” in IPFS(well, not just in IPFS). In this part, we will dive fully into IPLD and discuss: : What is the philosophy behind it, Why do we need it and Where does it fit into IPFS? The Significance of IPLD : An Explanation of its specification and how does it coordinate with other components of IPFS? How does IPLD Work? : Well, It’s always fun to play 😊 Playing with IPLD You can follow me on twitter @vasa_develop If you like high-tech Web3 concepts like IPLD explained in simple words with interactive tutorials, then . Hope you learn a lot about IPFS from this series. Let’s start! head here The Significance of IPLD IPLD is not just a part of the IPFS project, but a . To understand its significance in the decentralized world we have to understand the concept of . separate project in itself Linked Data: The Semantic Web What is Linked Data & Why do we Need it? The Semantic Web or Linked Data is a term coined by Sir Tim Berners Lee in a seminal article published in Scientific American in 2001. Berners Lee articulated a vision of a World Wide Web of data that machines could process independently of humans, enabling a host of new services transforming our everyday lives. While the paper’s vision of most web pages containing structured data that could be analyzed and acted upon by software agents , the Semantic Web has emerged as a platform of increasing importance for data interchange and integration through the growing community implementing data sharing using international , called . has NOT materialized semantic web standards Linked Data There are many current examples of the use of semantic web technologies and Linked Data to share valuable structured information in a flexible and extensible manner across the web. Semantic Web technologies are used extensively in the life sciences to facilitate drug discovery by finding paths across multiple datasets showing via genes linked to each. associations between drugs and side effects The New York Times has published its vocabulary of approximately 10,000 subject headings developed over 150 years as Linked Data and will expand coverage to approximately 30,000 topic tags; they encourage the development of services consuming these vocabularies and linking them with other online resources. The to make content more findable by search engines and more linkable through social media; to add additional context from supplemental resources in domains like music or sports, and to propagate linkages and editorial annotations beyond their original entry target to bring relevant information forward in additional contexts. British Broadcasting Corporation uses Linked Data The home page of the United States site states, “As the web of linked documents evolves to include the Web of linked data, we’re working to maximize the potential of Semantic Web technologies to realize the promise of Linked Open Government Data.” And also all the social media websites use Linked Data to create a web of People to make their platform as engaging as possible. data.gov So we do have some Linked Data but we still have a long way to go in order to harness the true power of Linked Data. Now imagine if you could refer your latest commits in a git branch to a bitcoin transaction to timestamp your work. So, by linking your git commit, you can view the commit from your blockchain explorer. Or, if you could link your Ethereum contract to media on IPFS, perhaps modifying it and tracking its changes on each function execution. All this is possible using IPLD. IPLD is the data model of the content-addressable web(as discussed in ). part1 It allows us to treat all hash-linked data structures as subsets of a unified information space, unifying all data models that link data with hashes as instances of IPLD. Or in other words, IPLD is a set of standards and implementations for creating decentralized data-structures that are universally addressable and linkable. These structures allow us to do for data what URLs and links did for HTML web pages. Content addressing through hashes has become a widely-used means of connecting data in distributed systems, from the blockchains that run your favorite cryptocurrencies, to the commits that back your code, to the web’s content at large. Yet, whilst all of these tools rely on some common primitives, their specific underlying data structures are (as right now I can’t connect my git commit to a blockchain transaction). NOT interoperable IPLD is a single namespace for . Through IPLD, . all hash-inspired protocols links can be traversed across protocols, allowing you to explore data regardless of the underlying protocol How does IPLD Work? Before diving deeper into IPLD, Let’s see the properties of IPLD. Properties of IPLD The sky’s the limit as IPLD allows you to work across protocol boundaries. The point is that IPLD provides that make the underlying data interoperable across tools and across protocols by default. libraries A canonical data model A that uniquely identifies any hash-based data structure and ensures the same logical object always maps to the exact same sequence of bits. self-contained descriptive model Protocol independent resolution IPLD brings isolated systems together(like connecting Bitcoin, Ethereum and git), making integration with existing protocols simple. Upgradeable With Multiformats(we will dive more into this in part 4) support, IPLD is easily upgradeable and will grow with your favorite protocols. Operates across formats Express your IPLD objects in various serializable formats like JSON, CBOR, YAML, XML and many more, making IPLD a cinch to use with any framework. Backward compatible Non-intrusive resolvers make IPLD easy to integrate within your existing work. A namespace for all protocols IPLD allows you to explore data across protocols seamlessly, binding hash-based data structures together through a common namespace. Now, let’s dive deeper into the IPLD. Diving Deeper into IPLD Specs IPLD is not a single specification, it is a set of specifications. (IPLD Stack) The goal of this stack is to enable decentralized data-structures which in turn will enable more decentralized applications. Many of the specifications in this stack are inter-dependent. (Dependency Graph) These diagrams show a high level of project specification of IPLD. Even if you don’t understand it fully it’s totally OK. To learn more about IPLD, here is a by great talk Juan Benet . OK. Enough of theoretical stuff. Let’s dive into the most fun part of this post 😊 Playing with IPLD In IPFS, IPLD helps to structure and link all the data chunks/objects. So, as we saw in , IPLD was responsible for organizing all the data chunks that constituted the image of the kitty🐱. part 1 In this part, we will create a medium.com like publication system, and link the tags, articles, and authors using IPLD. This will help you to get a more intuitive understanding of IPLD. You can also find the complete tutorial on . Github Let’s get started! Before creating the publication system, we’ll be exploring the IPFS DAG API, which lets us store data objects in IPFS in IPLD format. (You can store more exciting things in IPFS, like your favorite cat GIF, but we will stick to simpler things for now.) If you are not familiar with Merkle tries and DAGs then head over If you have an understanding of what these terms mean then continue… here . Create a folder name . Run and press for all the questions. ipld-blogs npm init Enter Now install the dependencies using: npm install ipfs-http-client cids --save After installing the module your project structure will look like this: Creating an IPLD format node You can create a new node by passing a data object into the method, which returns a Content Identifier (CID) for the newly created node. ipfs.dag.put ipfs.dag.put({name: ‘vasa’}) A CID is an address for a block of data in IPFS that is derived from its content. Every time someone puts the same data into IPFS, they'll get back an identical CID to the one you got. If they put in instead, the CID will be different. {name: 'vasa'} {name: 'vAsa'} Paste this code in and run tut.js node tut.js ipfsClient = ( ); CID = ( ); ipfs = ipfsClient({ : , : , : }); ipfs.dag.put({ : }, { : , : }, (err, cid)=>{ (err){ .log( , err);
  } multihash = cid.multihash; cids = CID( , , multihash); .log(cids.toBaseEncodedString()); }); //Initiate ipfs and CID instance const require 'ipfs-http-client' const require 'cids' //Connecting ipfs instance to infura node. You can also use your local node. const new host 'ipfs.infura.io' port '5001' protocol 'https' /*
Creating an IPLD format node:
ipfs.dag.put(dagNode, [options], [callback])

For more information see: 
https://github.com/ipfs/interface-js-ipfs-core/blob/master/SPEC/DAG.md#ipfsdagputdagnode-options-callback
*/ name "vasa" format 'dag-cbor' hashAlg 'sha2-256' if console "ERR\n" //featching multihash buffer from cid object. const //passing multihash buffer to CID object to convert multihash to a readable format const new 1 'dag-cbor' //Printing out the cid in a readable format console //zdpuAujL3noEMamveLPQWJPY6CYZHhHoskYQaZBvRbAfVwR8S You will get this CID . We have successfully created an IPLD format node. zdpuAujL3noEMamveLPQWJPY6CYZHhHoskYQaZBvRbAfVwR8S Connecting IPLD objects One important feature of Directed Acyclic Graphs (DAGs) is the ability to link them together. The way you express links in the DAG store is with the of another node. ipfs CID For example, if we wanted one node to have a link called “foo” pointed to another CID instance previously saved as , it might look like: barCid {
  foo: barCid
} When we give a field a name and make its value a link to a CID, we call this a named link. Below is the code showing how to create a named link. ipfsClient = ( ); CID = ( ); ipfs = ipfsClient({ : , : , : }); { vasa = ipfs.dag.put({ : }); secondNode = ipfs.dag.put({ : vasa});
}

linkNodes(); //Initiate ipfs and CID instance const require 'ipfs-http-client' const require 'cids' //Connecting ipfs instance to infura node. You can also use your local node. const new host 'ipfs.infura.io' port '5001' protocol 'https' /*
Creating an IPLD format node:
ipfs.dag.put(dagNode, [options], [callback])
For more information see: 
https://github.com/ipfs/interface-js-ipfs-core/blob/master/SPEC/DAG.md#ipfsdagputdagnode-options-callback
*/ async ( ) function linkNodes let await name 'vasa' //Linking secondNode to vasa using named link. let await linkToVasa Reading Nested data using Links You can read data from deeply nested objects using path queries. allows queries using IPFS paths. These queries return an object containing the value of the query and any remaining path that was unresolved. ipfs.dag.get The cool thing about this API is that it can also traverse through links. Here is an example of how you can read nested data using links. ipfsClient = ( ); CID = ( ); ipfs = ipfsClient({ : , : , : }); { (err) { .error( + err)
    } { .log(result)
    }
} { vasa = ipfs.dag.put({ : }); secondNode = ipfs.dag.put({ : { : { : vasa
            }
        }
    }); multihash = secondNode.multihash; cids = CID( , , multihash); ipfs.dag.get(cids.toBaseEncodedString()+ , errOrLog); }
createAndFeatchNodes(); //Initiate ipfs and CID instance const require 'ipfs-http-client' const require 'cids' //Connecting ipfs instance to infura node. You can also use your local node. const new host 'ipfs.infura.io' port '5001' protocol 'https' ( ) function errOrLog err, result if console 'error: ' else console async ( ) function createAndFeatchNodes let await name 'vasa' //Linking secondNode to vasa using named link. let await publication authors authorName //featching multihash buffer from cid object. const //passing multihash buffer to CID object to convert multihash to a readable format const new 1 'dag-cbor' //Featching the value using links '/publication/authors/authorName/name' /* prints { value: 'vasa', remainderPath: '' } */ You can also explore your IPLD nodes using this cool IPLD explorer. Like, if I want to see this CID: , I will go to this link: zdpuAujL3noEMamveLPQWJPY6CYZHhHoskYQaZBvRbAfVwR8S Now, as we have explored IPFS DAG API we are ready to work with IPLD and create our publication system. ttps://explore.ipld.io/#/explore/zdpuAujL3noEMamveLPQWJPY6CYZHhHoskYQaZBvRbAfVwR8S Creating a Publication System We will create a simple blogging application. This blogging application can: Add a new Author IPLD object. An author will have 2 fields: name and profile(a tag line for your profile). Create a new Post IPLD object. A post will have 4 fields: author, content, tags and publication date-time. Read a Post using post CID. Below is the code implementation for the above goals. ipfsClient = ( ); CID = ( ); ipfs = ipfsClient({ : , : , : }); { newAuthor = ipfs.dag.put({ : name, : }); .log( +name+ + newAuthor); newAuthor;
} { post = ipfs.dag.put({ : author, : content, : tags, : ()
    }); .log( + author + + post); post;
} {
    ipfs.dag.get(postCID + , (err, post) => { (err) { .error( + err)
        } { .log( , post);
            ipfs.dag.get(postCID + , (err, author) => { (err) { .error( + err)
                } { .log( , author);
                }
              });
        }
    });
} {
    addNewAuthor( ).then( {
        createBlog(newAuthor, , [ , , , ]).then( readBlog(postCID))
      });
}

startPublication(); /*
PUBLICATION SYSTEM
Adding new Author
An author will have
-> name
-> profile
Creating A Blog
A Blog will have a:
-> author
-> content
-> tags
-> timeOfPublish
List all Blogs for an author
Read a Blog
*/ //Initiate ipfs and CID instance const require 'ipfs-http-client' const require 'cids' //Connecting ipfs instance to infura node. You can also use your local node. const new host 'ipfs.infura.io' port '5001' protocol 'https' //Create an Author async ( ) function addNewAuthor name //creating blog author object var await name profile "Entrepreneur | Co-founder/Developer @TowardsBlockChain, an MIT CIC incubated startup | Speaker | https://vaibhavsaini.com" console "Added new Author " ": " return //Creating a Blog async ( ) function createBlog author, content, tags //creating blog object var await author content tags timeOfPublish Date console "Published a new Post by " ": " return //Read a blog async ( ) function readBlog postCID "" if console 'Error while reading post: ' else console "Post Details\n" "/author" if console 'Error while reading post author: ' else console "Post Author Details\n" ( ) function startPublication "vasa" ( ) => newAuthor "my first post" "ipfs" "ipld" "vasa" "towardsblockchain" ( ) => postCID On running this code and it will first create an author via , which will return the authors CID. Then this CID will be passed to function which will return the . This will be used by function to fetch the post details. addNewAuthor createBlog postCID postCID readBlog You can create more complex applications using IPLD… Ok. that’s it for this part. If you have any question, then you can shoot them in the comments. I hope you have learned a lot of thing from this post. In the next post, we will dive into the naming System of the distributed web, IPNS. So stay tuned… Thanks to for the excellent explanation on DAGs. proto.school Thanks for reading ;) Learned something? Share this article and help others to find it. Hold down the clap button if you liked the content! It helps me gain exposure. About the Author is a Co-Founder of Vaibhav Saini TowardsBlockchain , an MIT Cambridge Innovation Center incubated startup. He works as Senior blockchain developer and has worked on several blockchain platforms including Ethereum, Quorum, EOS, Nano, Hashgraph, IOTA etc. He is a Speaker, Writer and a drop-out from . IIT Delhi Want to learn more? Check out my previous articles. ConsensusPedia: An Encyclopedia of 30+ Consensus Algorithms Getting Deep Into Ethereum: How Data Is Stored In Ethereum? ContractPedia: An Encyclopedia of 40+ Smart Contract Platforms A Beginner’s Ultimate Guide To DAGs Follow me on Twitter: @vasa_develop

IOTA

Discovery

Fetch

Target

Twitter

YouTube

Understanding IPFS in Depth(3/6): What is InterPlanetary Naming System(IPNS)?

Understanding IPFS in Depth(1/6): A Beginner to Advanced Guide

Invite me as a Speaker

Book a call

Read My Stories

Too Long; Didn't Read

Understanding IPFS in Depth(2/6): What is InterPlanetary Linked Data(IPLD)?

Understanding IPFS in Depth(2/6): What is InterPlanetary Linked Data(IPLD)?

About Author

Comments

TOPICS

THIS ARTICLE WAS FEATURED IN

Related Stories

10+ Resources Every IPFS Developer Should Know About

(1/100) Crypto Countdown: Golem

A Research Report on the Trader $JOE DeFi Platform

07/03/2018: Biggest Stories in the Cryptosphere

05/02/2018: Biggest Stories in the Cryptosphere

01/06/2018: Biggest Stories in the Cryptosphere

10+ Resources Every IPFS Developer Should Know About

(1/100) Crypto Countdown: Golem

A Research Report on the Trader $JOE DeFi Platform

07/03/2018: Biggest Stories in the Cryptosphere

05/02/2018: Biggest Stories in the Cryptosphere

01/06/2018: Biggest Stories in the Cryptosphere

Light-Mode

Classic

Newspaper

Minty

Dark-Mode

Neon Noir

Minty

HN StartUps