How to Build a Twitter Bot with IPFS Cluster

Till now, we have explored a number of fun examples using (IPFS), (IPLD) & . We have built , , & . InterPlanetary File System InterPlanetary Linked Data Libp2p Websites on IPFS Youtube on IPFS Online Publication on IPLD Chat Application on Libp2p In this tutorial we are going to build a Twitter Bot(or a Pinbot) using . IPFS Cluster You can see our Twitter bot in action! You can try it out yourself. 1. Follow the twitter account @simpleaswater_ so that bot can filter your tweets out. 2. Tweet a photo, gif, video or any media, and mention “@simpleaswater_” in your tweet. Just like below: The bot will reply with the links to your content stored on the IPFS Cluster Network. Cool, right? Let’s see how you can build your own twitter pinbot! In this tutorial, we are go through: An Intro to IPFS Cluster, its significance & use-cases. Setting up an IPFS Cluster Network. Setting up Twitter Developer Account. Building a Twitter bot. You can find the here: full code implementation simpleaswater/twitter-pinbot If you get stuck in any part or have any queries/doubts, then feel free to reach us out on our discord channel . Intro to IPFS Cluster Before setting up our own IPFS Cluster network, it’s good to spend a few minutes to understand why do we need IPFS Cluster and what is it. If you are new to then we would recommend you to check out this post first: InterPlanetary File System (IPFS) Understanding IPFS in Depth(1/6): A Beginner to Advanced Guide If you are familiar with IPFS, then you already know that IPFS aims to make the internet decentralized using content addressing approach. The way this works is similar to services like BitTorrent where every user does not just consume the data but also serves it to other people in the BitTorrent network. That means you can not only get the data(let’s say Star Wars movie) from anyone on the network who has it, but also share this data with anyone who asks for it on the network. But if all the people who have the Star Wars movie go offline, then you are out of luck. No movie for you 😔 Now to understand consider the following scenario: why do we need IPFS Cluster Let’s suppose you want to store your collection of favorite songs on the IPFS Network. You upload your playlist on the IPFS network and the people who also love the same songs can get the songs from you. Now, you may also have other things on your device that you want to keep(your Movies, vacation photos, etc.). So, you decide to remove the playlist from your device. After all, you can get your playlist from the people who downloaded it from you earlier. So, you are not worried about your songs getting lost, and go to sleep happily. One fine day, you want to get your songs back for your long workout session but are devastated to see that you can’t find your playlist on the IPFS Network 😨 But how can this happen? You saved it on the network, right? It turns out that all the people who saved the playlist are either offline or have deleted the songs, as they got bored with them. The problem was that our playlist was on a number of nodes(or devices) so that we could be sure that it would not get lost in the oblivion. not stored redundantly So, how do we ensure data availability and redundancy on IPFS? Enter IPFS Cluster. The purpose of IPFS Cluster is to make it easy for you to orchestrate/manage data across several IPFS peers by allocating, replicating and tracking a global pinset(list of saved data) distributed among multiple peers. This way you can build a network of IPFS peers which redundantly store data for us. If you want to know more about IPFS Cluster and see how it works under the hood, then head here: Complete IPFS Cluster Guide: Concepts, Tutorials & Examples If you have any queries/doubts till now, then feel free to reach us out on our discord channel . Now, as we have explored why we need IPFS Cluster let’s setup our own IPFS Cluster network, on which we will save our tweets. Setting up Your IPFS Peer As IPFS Cluster network is a collection of IPFS peers, first we need to install & run an IPFS peer. You can follow this guide to install IPFS Peer on your device. Installing & Updating IPFS on Windows, Mac & Linux After installing the IPFS, you can test your installation using ipfs version $ ipfs version ipfs version $ ipfs help USAGE: ipfs - Global p2p merkle-dag filesystem. ... NOTE: Throughout this tutorial, we use the $ character to indicate your terminal’s shell prompt. When following along, don’t type the $ character, or you’ll get some weird errors. Now, let’s initialize your IPFS peer using ` ` ipfs init $ ipfs init initializing IPFS node at /home/vasa/.ipfs generating 2048-bit RSA keypair...done peer identity: Qmcpo2iLBikrdf1d6QU6vXuNb6P7hwrbNPW9kLAH8eG67z to get started, enter: ipfs cat /ipfs/QmYwAPJzv5CZsnA625s3Xf2nemtYgPpHdWEz79ojWnPbdG/readme Voila! Your IPFS peer setup is complete! If you are having any problems while setting up your IPFS peer, then feel free to reach us out on our discord channel . Setting up Your IPFS Cluster Peer In order to run an IPFS Cluster peer we need to install . ipfs-cluster-service First, download the ipfs-cluster-service binary for your Operating System. Now, unzip/untar the downloaded binary and you will find a executable inside the folder. Place this binary where it can be run by a system user(usually ). ipfs-cluster-service usr/local/bin Now run the following command to initialize generate the IPFS Cluster configuration file. ipfs-cluster-service init Viola! You have setup your IPFS Cluster Peer. Making a few changes to IPFS Cluster Config In order to make the twitter bot example work, we need to change the IPFS Cluster config a bit. If you want to know why we are doing these changes, then you can refer to . this discussion thread To make the changes, first open your IPFS Cluster config located at . $HOME/.ipfs-cluster / service.json Now, scroll down to the api JSON object. In the api section, you will find restapi JSON object. You need to add these 3 key-values: , and as shown below. libp2p_listen_multiaddress private_key id { ... "api": { "ipfsproxy": { ... }, "restapi": { "libp2p_listen_multiaddress": "/ip4/127.0.0.1/tcp/9696", "private_key": "CAASqAkwggSkAgEAAoIBAQDLZZcGcbe4urMBVlcHgN0fpBymY+xcr14ewvamG70QZODJ1h9sljlExZ7byLiqRB3SjGbfpZ1FweznwNxWtWpjHkQjTVXeoM4EEgDSNO/Cg7KNlU0EJvgPJXeEPycAZX9qASbVJ6EECQ40VR/7+SuSqsdL1hrmG1phpIju+D64gLyWpw9WEALfzMpH5I/KvdYDW3N4g6zOD2mZNp5y1gHeXINHWzMF596O72/6cxwyiXV1eJ000k1NVnUyrPjXtqWdVLRk5IU1LFpoQoXZU5X1hKj1a2qt/lZfH5eOrF/ramHcwhrYYw1txf8JHXWO/bbNnyemTHAvutZpTNrsWATfAgMBAAECggEAQj0obPnVyjxLFZFnsFLgMHDCv9Fk5V5bOYtmxfvcm50us6ye+T8HEYWGUa9RrGmYiLweuJD34gLgwyzE1RwptHPj3tdNsr4NubefOtXwixlWqdNIjKSgPlaGULQ8YF2tm/kaC2rnfifwz0w1qVqhPReO5fypL+0ShyANVD3WN0Fo2ugzrniCXHUpR2sHXSg6K+2+qWdveyjNWog34b7CgpV73Ln96BWae6ElU8PR5AWdMnRaA9ucA+/HWWJIWB3Fb4+6uwlxhu2L50Ckq1gwYZCtGw63q5L4CglmXMfIKnQAuEzazq9T4YxEkp+XDnVZAOgnQGUBYpetlgMmkkh9qQKBgQDvsEs0ThzFLgnhtC2Jy//ZOrOvIAKAZZf/mS08AqWH3L0/Rjm8ZYbLsRcoWU78sl8UFFwAQhMRDBP9G+RPojWVahBL/B7emdKKnFR1NfwKjFdDVaoX5uNvZEKSl9UubbC4WZJ65u/cd5jEnj+w3ir9G8n+P1gp/0yBz02nZXFgSwKBgQDZPQr4HBxZL7Kx7D49ormIlB7CCn2i7mT11Cppn5ifUTrp7DbFJ2t9e8UNk6tgvbENgCKXvXWsmflSo9gmMxeEOD40AgAkO8Pn2R4OYhrwd89dECiKM34HrVNBzGoB5+YsAno6zGvOzLKbNwMG++2iuNXqXTk4uV9GcI8OnU5ZPQKBgCZUGrKSiyc85XeiSGXwqUkjifhHNh8yH8xPwlwGUFIZimnD4RevZI7OEtXw8iCWpX2gg9XGuyXOuKORAkF5vvfVriV4e7c9Ad4Igbj8mQFWz92EpV6NHXGCpuKqRPzXrZrNOA9PPqwSs+s9IxI1dMpk1zhBCOguWx2m+NP79NVhAoGBAI6WSoTfrpu7ewbdkVzTWgQTdLzYNe6jmxDf2ZbKclrf7lNr/+cYIK2Ud5qZunsdBwFdgVcnu/02czeS42TvVBgs8mcgiQc/Uy7yi4/VROlhOnJTEMjlU2umkGc3zLzDgYiRd7jwRDLQmMrYKNyEr02HFKFn3w8kXSzW5I8rISnhAoGBANhchHVtJd3VMYvxNcQb909FiwTnT9kl9pkjhwivx+f8/K8pDfYCjYSBYCfPTM5Pskv5dXzOdnNuCj6Y2H/9m2SsObukBwF0z5Qijgu1DsxvADVIKZ4rzrGb4uSEmM6200qjJ/9U98fVM7rvOraakrhcf9gRwuspguJQnSO9cLj6", "id": "QmcrQZ6RJdpYuGvZqD5QEHAv6qX4BrQLJLQPQUrTrzdcgm", ... } } ... } NOTE: The private_key that we are using is just for the demo purpose. If you are building a production bot, you should create your own libp2p private_key and id. Starting Your IPFS Cluster Network Now, as we have installed IPFS and IPFS Cluster peers, we can start our Cluster network. 1. Open a terminal window and start the IPFS peer using . ipfs daemon 2. Open and start the IPFS Cluster peer using . another terminal ipfs-cluster-service daemon If you managed to reach here, then you deserve a pat on your back! You are now running a IPFS Cluster network with 1 peer. For the purpose of this tutorial we will proceed with this setup. In case, you want to setup a more complex production infrastructure, then you can refer to . IPFS Cluster docs If you have any queries/doubts, then feel free to reach us out on our discord channel . Creating a Twitter Developer Account To build a twitter bot, first we need to setup a developer account on twitter. 1. Go to . Click the “Create an app” button. Twitter developers website 2. You will be prompted to apply for the twitter developer account. 3. Now, select the “Making a bot” option for your primary reason for using twitter developer tools, and click “Next”. 4. In the next page, you will be asked some personal details. Fill them up and click “Next”. 5. In the next page you will be asked a few more questions about how will you use twitter’s APIs. Fill up the answers by saying that you are creating a bot for fun 😉. After filling all the info, click “Next”. 6. In the last page, you will be asked to confirm the details that you provided. 7. You may have to confirm your email account. And wait for sometime to get your account approved. After your account has been approved, you can go your , and click “Create an app”, fill the app details and hit “Create” button. twitter apps page Congratulations! You now have a twitter app 🎉 Now, come back to the , where you can see your app. Click on the “details” button. twitter apps page Navigate to the “keys and tokens” section, where you will find all the credentials we want. Copy the “API key”, “API secret key”, “Access token” & “Access token secret”. Building the Twitter Bot 💎 Now, as we have everything we need to build the Twitter Pinbot, let’s fire up our code editor! Installing Golang We are going to use to build our Twitter bot. You can . Golang download Golang from here First you need to .You will be redirected to a page with installation instructions. Follow these instructions to install Golang. download Golang After installing Golang, we are ready to start with the twitter bot code. Writing the Twitter Bot Code 👩💻👨💻 We are going to build a twitter bot which works in the following way: You need to follow the twitter bot account, so that the bot can filter out your tweets from all the tweets.Then you can “pin”, “unpin” or “add” data to the IPFS Cluster Network by tweeting to with some specific text. @botHandle //To pin a CID with a name/label to IPFS Cluster Network @botHandle !pin //To unpin a CID from IPFS Cluster Network @botHandle !unpin //To add a file(via URL) to IPFS Cluster Network @botHandle !add //To check out what the bot can do @botHandle !help //Adding photos, memes, videos to IPFS Cluster Network Tweet to @botHandle attaching photos, memes or videos Now, let’s start by downloading some boilerplate code. Clone the branch of repo: boilerplate twitter-pinbot $ git clone --single-branch --branch boilerplate https://github.com/simpleaswater/twitter-pinbot Now, first we need to make a few changes in the file. config.json { : , : , : , : , : , : , : , : , : } "twitter_name" "@botHandle" "twitter_id" "@botHandle" "consumer_key" "API key" "consumer_secret" "API secret key" "access_key" "Access token" "access_secret" "Access token secret" "cluster_peer_addr" "cluster peer multiaddress" "cluster_username" "" "cluster_password" "" You need to add your and the credentials that we got from the twitter developer portal. @botHandle Then replace the with: “cluster peer multiaddress” /ip4/127.0.0.1/tcp/9696/ipfs/QmcrQZ6RJdpYuGvZqD5QEHAv6qX4BrQLJLQPQUrTrzdcgm If you look closely, you can see that this is the same multiaddress which we added above while changing the IPFS Cluster config file. We do this to interact with the IPFS Cluster Setup that we created above. You can leave the & blank. “cluster_username” “cluster_password” Now, as we are done with our file, let’s start working on config.json main.go. main ( ) ConfigFile = IPFSGateway = twittercom = Action ( actionRegexp = regexp.MustCompile( ) pinRegexp = regexp.MustCompile( ) PinAction Action = UnpinAction Action = AddAction Action = HelpAction Action = ) { } package import "regexp" // ConfigFile is the path of the default configuration file var "config.json" // Gateway var "https://ipfs.io" const "twitter.com" type string // Variables containing the different available actions var // (spaces)(action)whitespaces(arguments) `^\s*([[:graph:]]+)\s+(.+)` // (cid)whitespaces(name with whitespaces). [:graph:] does not // match line breaks or spaces. `([[:graph:]]+)\s+([[:graph:]\s]+)` "!pin" "!unpin" "!add" "!help" func main () //Let's code ☕ The file contains a few constants that we will be using as we move forward in the tutorial. main.go Let’s first add a way to read and manage the data from config.json. Config { TwitterID TwitterName AccessKey AccessSecret ConsumerKey ConsumerSecret ClusterPeerAddr ClusterUsername ClusterPassword } { cfg := &Config{} cfgFile, err := ioutil.ReadFile(path) err != { log.Fatal(err) } err = json.Unmarshal(cfgFile, &cfg) err != { log.Fatal(err) } cfg } // Config is the configuration format for the Twitter Pinbot type struct string `json:"twitter_id"` string `json:"twitter_name"` string `json:"access_key"` string `json:"access_secret"` string `json:"consumer_key"` string `json:"consumer_secret"` string `json:"cluster_peer_addr"` string `json:"cluster_username"` string `json:"cluster_password"` // Function to read JSON config file * func readConfig (path ) string Config if nil if nil return Here we have struct which manages different key-value pairs from our file. Also, we have the function which takes the path of the and parses the from the file. Config config.json readConfig(path string) config.json JSON Next, we create the struct, which is used to manage the different functionalities of the twitter pinbot. Bot Bot { ctx context.Context cancel context.CancelFunc name id twClient *twitter.Client clusterClient client.Client followedBy sync.Map die {} } // Bot is a twitter bot which reads a user's timeline // and performs actions on IPFS Cluster if the tweets // match, i.e. a tweet with: "@botHandle !pin " // will pin something. The users with pin permissions are // those who follow the bot. Retweets by users who follow // the bot should also work. The bot will answer // the tweet with a result. type struct string string chan struct You will also have to import these 2 modules: ( ) import "github.com/dghubble/go-twitter/twitter" "github.com/ipfs/ipfs-cluster/api/rest/client" Now, let’s add the function to create a . Bot ( ) func New(cfg *Config) (*Bot, error) { ctx, := context.WithCancel(context.Background()) ocfg := oauth1.NewConfig(cfg.ConsumerKey, cfg.ConsumerSecret) token := oauth1.NewToken(cfg.AccessKey, cfg.AccessSecret) httpClient := ocfg.Client(ctx, token) twClient := twitter.NewClient(httpClient) peerAddr, := multiaddr.NewMultiaddr(cfg.ClusterPeerAddr) err != nil { cancel() nil, err } clusterClient, := client.NewDefaultClient(&client.Config{ : peerAddr, : cfg.ClusterUsername, : cfg.ClusterPassword, : , }) err != nil { cancel() nil, err } bot := &Bot{ : ctx, : cancel, : twClient, : clusterClient, : cfg.TwitterName, : cfg.TwitterID, : make(chan struct{}, ), } bot.fetchFollowers() go bot.watchFollowers() go bot.watchTweets() bot, nil } import "github.com/multiformats/go-multiaddr" "github.com/dghubble/oauth1" // New creates a new Bot with the Config. cancel // Creating Twitter client // Creating IPFS Cluster client err if return err APIAddr Username Password LogLevel "info" if return //Creating Bot Object ctx cancel twClient clusterClient name id die 1 return Here we first create which interacts with the twitter APIs. Also, we create which interacts with our IPFS Cluster Setup. Then, using these 2, we create our object. twClient clusterClient Bot At last, we have 3 methods: : This method is used to fetch the list of bot followers, so that we can track their tweets, and respond accordingly. fetchFollowers() : This is used to watch the list of bot followers every 60 seconds. In case, somebody follows the bot, the list will be updated within 60 minutes, after which we will start filtering the tweets of the recent follower. watchFollowers() goroutine : This is used to watch & filter the tweets who mention our watchTweets() goroutine @botHandle. We will implement these 3 methods in a while. Next, we have a few small utility functions. { b.cancel() } { b.name } { b.id } // Kill destroys this bot. func (b *Bot) Kill () // Name returns the twitter handle used by the bot func (b *Bot) Name () string return // ID returns the twitter user ID used by the bot func (b *Bot) ID () string return As explained in the comments: : destroys the bot. Kill() : returns the twitter handle used by the bot. Name() : returns the twitter user ID used by the bot. ID() Now, let’s implement the above 3 methods. { nextCursor = includeEntities := nextCursor != { followers, _, err := b.twClient.Followers.List( &twitter.FollowerListParams{ Count: , IncludeUserEntities: &includeEntities, }) err != { log.Println(err) } _, u := followers.Users { _, old := b.followedBy.LoadOrStore(u.ID, {}{}) !old { log.Println( , u.ScreenName) } } nextCursor = followers.NextCursor time.Sleep( * time.Second) } } //Fetching the bot Followers func (b *Bot) fetchFollowers () var int64 -1 false for 0 //Getting the List of bot followers 200 //Number of bot followers we want in our list if nil //Parsing and Storing the ID(s) of the bot followers for range struct if //Printing the bot followers ScreenName "Friend: " 2 The method fetches the list of bot followers using . You can specify a maximum cap( ) on how many latest followers will be allowed to interact with the bot. We will set this to 200. You can change this as per your wish. fetchFollowers() twClient Count We also parse through the list to get the of our bot followers, and print out their screen-names. followers ID Next, let’s implement the method. watchFollowers() { { time.Sleep( * time.Second) { <-b.ctx.Done(): : b.fetchFollowers() } } } //Watching bot followers every 60 secs. //This checks the accounts following //the bot in every 60 seconds, so that //any new follower can be added to the //followers list. func (b *Bot) watchFollowers () for 60 select case default As the comments suggest, we call the method every 60 seconds to check for new bot followers and update our list. fetchFollowers() followers Now, let’s implement the method. watchTweets() { log.Println( ) params := &twitter.StreamFilterParams{ Track: [] { PinAction.String(), UnpinAction.String(), HelpAction.String(), AddAction.String(), b.Name(), }, StallWarnings: twitter.Bool( ), } stream, err := b.twClient.Streams.Filter(params) err != { log.Println(err) } demux := twitter.NewSwitchDemux() demux.Tweet = { b.processTweet(t, t) } { { <-b.ctx.Done(): msg := " strings in tweet body. */ string true if nil /* Receiving messages of type interface{} isn't very nice, it means you'll have to type switch and probably filter out message types you don't care about. For this,we use Demux, which receives messages and type switches them to call functions with typed messages. For example, say we're only interested in Tweets. */ func (t *twitter.Tweet) //Processing the tweets //Constantly watching for new filtered tweets for select case return case //"Handle(msg)" determines the type of a message //and calls the corresponding receiver //function with the typed message. go func (a Action) String () string return string The method continuously watches for different tweets, by adding a filter on the tweets. We filter for specific strings occurrence in the tweets, so that we don't reply to the every tweet by who mentions the . watchTweets() @botHandle After filtering the tweets, we process the tweets using . processTweet(t, t) ( ) { tweet == { } srcTweet == { srcTweet = tweet } tweet.User.IDStr == b.ID() || srcTweet.User.IDStr == b.ID() { } action, arguments, urls, err := b.parseTweet(tweet) err != { b.tweet(err.Error(), tweet, srcTweet, ) } log.Printf( , action, arguments, urls) _, ok := b.followedBy.Load(srcTweet.User.ID) !ok && action.Valid() { log.Println( ) b.tweet( , tweet, srcTweet, ) } !ok { } action { PinAction: b.pin(arguments, tweet, srcTweet) UnpinAction: b.unpin(arguments, tweet, srcTweet) AddAction: b.add(arguments, tweet, srcTweet) HelpAction: b.tweetHelp(tweet, srcTweet) : log.Println( ) } (urls) > { log.Println( , urls) out := ( *api.AddedOutput, ) { cids := []cid.Cid{} added := out { log.Printf( , added.Cid) cids = (cids, added.Cid) } (cids) > { b.tweetAdded(cids, tweet, srcTweet) } }() params := api.DefaultAddParams() params.Wrap = params.Name = + tweet.IDStr err := b.clusterClient.Add(context.Background(), urls, params, out) err != { log.Println(err) } } retweets := []*twitter.Tweet{tweet.QuotedStatus, tweet.RetweetedStatus} _, rt := retweets { b.processTweet(rt, srcTweet) } } import "github.com/ipfs/go-cid" "github.com/ipfs/ipfs-cluster/api" //Process the filetered tweets and handle the tweets according to the //desired action. func (b *Bot) processTweet (tweet *twitter.Tweet, srcTweet *twitter.Tweet) if nil return if nil // Skip processing our own tweets (written by us) // and quotes or retweets we've made (origUser is us) // (avoid potential loops) if return if nil false return "Parsed: %s, %s, %s\n" if "Error: NoFollow" "Follow me, and try again." false return if return // Process actions switch case //Pin the Tweet to the IPFS Cluster network case //UnPin the Tweet to the IPFS Cluster network case ////Add the Tweet to the IPFS Cluster network case //Tweet a "help tweet" that demostrates how to use the bot default //Need to handle default tweets by adding the assets "no handled action for this tweet" // Add any media urls if len 0 "adding media: " make chan 1 go func () for range "added %s\n" append if len 0 true "Tweet-" if nil // If the tweet has retweets, process them as if they were // from this user. for range The method processes the filtered tweets and handles the tweets according to the desired action. processTweet(tweet, tweet) First, we check if the tweet is originated from the bot’s account or not. If the tweet originated from the bot’s account then we , otherwise we will end up in an infinite loop, tweeting our own tweet. return Then we parse the tweets using to get the , & from the tweet. We will implement the in a while. parseTweet(tweet) action arguments urls parseTweet(tweet) After that we check 2 things. First, we check that the tweets must originate from accounts that follow the bot. Second, we check if the action is a valid action or not. We can check the validity of the parsed using the following function. action { a { PinAction, UnpinAction, AddAction, HelpAction: } } func (a Action) Valid () bool switch case return true return false If the tweet is not from someone who follows the bot, then we return. Then, in the statement, we check for type and perform different actions accordingly. We will implement these different methods( , , etc.) in a while. switch action pin unpin After that we check for any , and use to the to the IPFS Cluster setup, and return the . urls clusterClient Add urls content identifiers (CIDs) And finally, we handle an edge case in which if the tweet has retweets, then we process them as if they were from the bot follower, and thus calling the . processTweet(rt, srcTweet) Now, let’s implement the method. parseTweet(tweet) { tweet.ExtendedTweet != { tweet.Entities = tweet.ExtendedTweet.Entities tweet.ExtendedEntities = tweet.ExtendedTweet.ExtendedEntities tweet.FullText = tweet.ExtendedTweet.FullText } text := tweet.FullText text == { text = tweet.Text } log.Println( , text) text = strings.TrimPrefix(text, b.name) action Action arguments text == + (HelpAction) { HelpAction, , [] {}, } matches := actionRegexp.FindAllStringSubmatch(text, ) (matches) > { firstMatch := matches[ ] action = Action(firstMatch[ ]) arguments = firstMatch[ ] } urls := extractMediaURLs(tweet) action, arguments, urls, } // parseTweet returns Action, arguments, media urls, and error func (b *Bot) parseTweet (tweet *twitter.Tweet) (Action, , [] , error) string string // Extended tweet? let's use the entities from the extended tweet then. if nil if "" "Parsing:" // remote our username if they started with it var var string if " " string return "" string nil // match to see if any action -1 if len 0 0 1 // first group match 2 // second group match return nil Here, we first handle the , which is a enterprise feature of Twitter, in which you can can have tweets with no word limit. ExtendedTweet type Then we apply to our tweets to check for & . For extracting the urls we use . regular expression action arguments extractMediaURLs(tweet) { urls [] _, m := media(tweet.ExtendedEntities) { urls = (urls, extractMediaURL(&m)) } (urls) == { _, m := media(tweet.Entities) { urls = (urls, extractMediaURL(&m)) } } urls } { ent == { } ent.( ) { *twitter.Entities: e := ent.(*twitter.Entities) e != { e.Media } *twitter.ExtendedEntity: e := ent.(*twitter.ExtendedEntity) e != { e.Media } } } //Extracting MediaURLs from tweets [] func extractMediaURLs (tweet *twitter.Tweet) string var string // Grab any media entities from the tweet for range append if len 0 // If no extended entitites, try with traditional. for range append return // takes *Entities or *MediaEntities [] . func media (ent {}) interface twitter MediaEntity if nil return nil switch type case if nil return case if nil return return nil Here we loop through the and to grab the media entities. Now, we extract the actual using the function. ExtendedEntities Entities MediaURL extractMediaURL(MediaEntity) byBitrate []twitter.VideoVariant { (vv) } { vv[i], vv[j] = vv[j], vv[i] } { vv[i].Bitrate < vv[j].Bitrate } { me.Type { , : variants := me.VideoInfo.Variants sort.Sort(byBitrate(variants)) last := variants[ (variants) ] last.URL : me.MediaURL } } type func (vv byBitrate) Len () int return len func (vv byBitrate) Swap (i, j ) int func (vv byBitrate) Less (i, j ) int bool return //Extracting the highest bitrate MediaURL from MediaEntity func extractMediaURL (me *twitter.MediaEntity) string switch case "video" "animated_gif" // pick video with highest bitrate len -1 return default return extracts the with the highest bitrate from the available variants of the or URLs. extractMediaURL(MediaEntity) MediaURL video animated_gif Now, as we have parsed all the necessary information from the filtered tweets, we can proceed to implement , , and methods. pin unpin add { log.Println( , args) pinUsage := fmt.Sprintf( , PinAction) matches := pinRegexp.FindAllStringSubmatch(args, ) (matches) == { b.tweet(pinUsage, srcTweet, , ) } firstMatch := matches[ ] cidStr := firstMatch[ ] name := firstMatch[ ] c, err := cid.Decode(cidStr) err != { b.tweet(pinUsage+ , tweet, srcTweet, ) } _, err = b.clusterClient.Pin(context.Background(), c, api.PinOptions{Name: name}) err != { log.Println(err) b.tweet( , srcTweet, , ) b.die <- {}{} } waitParams := client.StatusFilterParams{ Cid: c, Local: , Target: api.TrackerStatusPinned, CheckFreq: * time.Second, } ctx, cancel := context.WithTimeout(b.ctx, *time.Minute) cancel() _, err = client.WaitFor(ctx, b.clusterClient, waitParams) err != { log.Println(err) b.tweet( , srcTweet, , ) } b.tweet(fmt.Sprintf( , IPFSGateway, cidStr), tweet, srcTweet, ) } //Function to pin a CID to IPFS Cluster func (b *Bot) pin (args , tweet, srcTweet *twitter.Tweet) string "pin with " "Usage: '%s '" -1 if len 0 nil false return 0 1 2 if nil ". Make sure your CID is valid." false return if nil "An error happened pinning. I will re-start myself. Please retry in a bit." nil false struct return false 10 10 defer if nil "IPFS Cluster has been pinning this for 10 mins. This is normal for big files. Otherwise, make sure there are providers for it. Don't worry, Cluster will keep at it for a week before giving up." nil false return "Pinned! Check it out at %s/ipfs/%s" true Here we again apply regular expression on to validate the tweet text and content identifier( ) and name( ) for the pin to be added to IPFS Cluster setup. args cidStr name We have added a number of error checks for: Checking of the CID( ) is valid. cidStr Checking if some unexpected error happened while pinning the CID to IPFS Cluster setup using . clusterClient.Pin Checking if the pinning process takes more that 10 seconds(timeout), which is sufficient for big files. In case, everything is fine, the bot tweets the link for the added pin using the method. We will implement the method in a while. IPFSGateway tweet tweet Now, let’s see the method. unpin { log.Println( , args) unpinUsage := fmt.Sprintf( , UnpinAction) c, err := cid.Decode(args) err != { b.tweet(unpinUsage+ , tweet, srcTweet, ) } _, err = b.clusterClient.Unpin(context.Background(), c) err != && !strings.Contains(err.Error(), ) { log.Println(err) b.tweet( , srcTweet, , ) b.die <- {}{} } waitParams := client.StatusFilterParams{ Cid: c, Local: , Target: api.TrackerStatusUnpinned, CheckFreq: * time.Second, } ctx, cancel := context.WithTimeout(b.ctx, time.Minute) cancel() _, err = client.WaitFor(ctx, b.clusterClient, waitParams) err != { log.Println(err) b.tweet( , srcTweet, , ) } b.tweet(fmt.Sprintf( , args), tweet, srcTweet, ) } // Function to unpin a CID from IPFS Cluster network func (b *Bot) unpin (args , tweet, srcTweet *twitter.Tweet) string "unpin with " "Usage: '%s '" if nil ". Make sure your CID is valid." false return if nil "uncommited to state" "An error happened unpinning. I will re-start myself. Please retry in a bit." nil false struct return false 10 defer if nil "IPFS Cluster did not manage to unpin the item, but it's trying..." nil false return "Unpinned %s! :'(" false In the method we use to get the CID( ) from the . Similar to , here also we have a few checks in place to handle errors. unpin cid.Decode c args pin Making sure that the CID( ) is valid. c Checking if some unexpected error happened while unpinning the CID from IPFS Cluster setup using . clusterClient.Unpin Checking if the unpinning process takes more that 10 seconds(timeout). In case, everything is fine, the bot tweets the acknowledging that the CID has been unpinned. Now, let’s see the method. add { log.Println( , arg) addUsage := fmt.Sprintf( ) url, err := url.Parse(arg) err != { b.tweet(addUsage+ , srcTweet, , ) } url.Scheme != && url.Scheme != { b.tweet(addUsage+ , srcTweet, , ) } url.Host == || url.Host == || url.Host == { b.tweet( , srcTweet, , ) } out := ( *api.AddedOutput, ) { cids := []cid.Cid{} added := out { cids = (cids, added.Cid) } (cids) > { b.tweetAdded(cids, tweet, srcTweet) } }() params := api.DefaultAddParams() params.Wrap = params.Name = + tweet.IDStr log.Println([] {arg}) err = b.clusterClient.Add(context.Background(), [] {arg}, params, out) err != { log.Println(err) b.tweet( , srcTweet, , ) b.die '" if nil ". Make sure you gave a valid url!" nil false return if "http" "https" ". Not an HTTP(s) url!" nil false return if "localhost" "127.0.0.1" "::1" "ehem ehem..." nil false return make chan 1 go func () for range append if len 0 true "Tweet-" string string if nil "An error happened adding. I will re-start myself. Please retry in a bit." nil false struct return The method is used to add data via URLs in the tweet to the IPFS Cluster setup. We parse the using . As the above 2 methods, also has checks to counter catch the errors. add url url.Pars(arg) add Making sure that the URL( ) is valid. c Checking if the and are valid. url.Scheme url.Host Checking if there is an unexpected error while adding files to the IPFS Cluster setup. In case, everything is fine, the bot tweets CID(s) of the file(via the URL) and the folder-wrap(file wrapped in a folder) using . tweetAdded { msg := i, c := cids { i != (cids) { msg += fmt.Sprintf( , IPFSGateway, c) } { msg += fmt.Sprintf( , IPFSGateway, c) } } b.tweet(msg, tweet, srcTweet, ) } //Function for Tweeting for the Add Action func (b *Bot) tweetAdded (cids []cid.Cid, tweet, srcTweet *twitter.Tweet) "Just added this to #IPFS Cluster!\n\n" for range if len -1 "• File: %s/ipfs/%s\n" else // last "• Folder-wrap: %s/ipfs/%s\n" true Here is the implementation of the method which tweets how to use the bot. tweetHelp { help := fmt.Sprintf( , b.name) b.tweet(help, srcTweet, , ) } //Function for Tweeting for the Help Action func (b *Bot) tweetHelp (tweet, srcTweet *twitter.Tweet) `Hi! Here's what I can do: !pin !unpin !add !help You can always prepend these commands mentioning me (%s). Happy pinning! ` nil false Now, let’s implement the method which sends tweets quoting or replying to the filtered tweets. tweet { tweetMsg := params := &twitter.StatusUpdateParams{} sameTweets := inReplyTo == { tweetMsg = msg TWEET } sameTweets = srcTweet == || inReplyTo.ID == srcTweet.ID params.InReplyToStatusID = inReplyTo.ID { sameTweets && !quote: tweetMsg = fmt.Sprintf( , inReplyTo.User.ScreenName, msg) sameTweets && quote: tweetMsg = fmt.Sprintf( , inReplyTo.User.ScreenName, msg, permaLink(inReplyTo), ) !sameTweets && !quote: tweetMsg = fmt.Sprintf( , inReplyTo.User.ScreenName, srcTweet.User.ScreenName, msg, ) !sameTweets && quote: tweetMsg = fmt.Sprintf( , srcTweet.User.ScreenName, msg, permaLink(inReplyTo), ) } TWEET: log.Println( , tweetMsg) newTweet, _, err := b.twClient.Statuses.Update(tweetMsg, params) err != { log.Println(err) } _ = newTweet } { fmt.Sprintf( , twittercom, tweet.User.ScreenName, tweet.IDStr) } // tweets sends a tweet quoting or replying to the given tweets. // srcTweet might be nil. // Otherwise it just posts the message. func (b *Bot) tweet (msg , inReplyTo, srcTweet *twitter.Tweet, quote ) string bool "" false if nil goto nil switch case // @user msg (reply thread) "@%s %s" case // @user msg (quote RT) ".@%s %s %s" case // @user @srcUser msg (reply thread) "@%s @%s %s" case // @srcuser (quote RT mentioning src user) ".@%s %s %s" "tweeting:" if nil return // if quote { // then retweet my tweet after a minute // go func() { // time.Sleep(time.Minute) // _, _, err := b.twClient.Statuses.Retweet(newTweet.ID, nil) // log.Println("retweeted: ", tweetMsg) // if err != nil { // log.Println(err) // return // } // }() // } return func permaLink (tweet *twitter.Tweet) string return "https://%s/%s/status/%s" Now, finally, let’s implement the function of our pinbot, which will complete our pinbot code. main { path := flag.String( , ConfigFile, ) flag.Parse() cfg := readConfig(*path) bot, err := New(cfg) err != { log.Fatal(err) } log.Println( , bot.Name(), bot.ID()) ch := ( os.Signal) signal.Notify(ch, syscall.SIGINT, syscall.SIGTERM) { sig := <-ch: log.Println(sig) <-bot.die: } bot.Kill() } func main () //Fetching the optional path from command line "config" "path to config file" //Reading the config file //Creating a new bot if nil "Bot created:" // Wait for SIGINT and SIGTERM (HIT CTRL-C) make chan select case case In the main function, we read the config file from the using the function. Then using the config file key-values( ) we create our bot using the function. path readConfig cfg New(cfg) Also, we wait for and so that hitting kills out . SIGINT SIGTERM CTRL-C bot Final Showdown Now, it’s time to test our code. First build the code by executing in the directory/folder( ) where you have saved your file. go build twitter-pinbot main.go This will download all the modules that we have imported. After this is done, you will see an executable file named twitter-pinbot. Before starting, make sure that , IPFS dameon and IPFS Cluster daemon are running, that we started above. both Now, you can run the executable file by executing . ./twitter-pinbot If everything works fine, then you would see a list of your bot followers like this: Now, we are waiting for your tweets… If you get stuck in any part or have any queries/doubts, then feel free to reach us out on our discord channel . This article was first published on our open-source platform, SimpleAsWater.com . If you are interested in IPFS, Libp2p, Ethereum, Zero-knowledge Proofs, Defi, CryptoEconomics, IPLD, Multi formats, and other Web 3.0 projects, concepts and interactive tutorials, then be sure to check out SimpleAsWater .