This is the first part of a X part article. Part 2 Part 3 Part 4 First of all a question. How do you find out about a fake viral article? Does somebody tell you abou t it? Are you using a special software? Do you use facebook trends and twitter trends to find out what is viral? To find out how journalists find out in the present, i made this I encourage you to so we can get a better understanding of how journalists and bloggers find out about viral news articles. google form . complete the form What we need to build to acomplish this: The Outbrake is a service that i want to developed for Journalists, allowing them to find viral articles before they go viral. Idea By automatic crawling the FB pages that usually share fake or misleading articles, journalists can see the “next lies” before they are already so popular so even if you will explain that something is not true, more people had heared the fake version. Every article that now have , in the had , in the , the 600K Shares first hour 5K shares second 11K third hour 19K Using this, we can detect this articles way before they spread from one vertical into another and they become viral. The highlevel details Every hour, we look at what news articles post the and download them into a database, searching for the ones that are getting viral. top 1000 most popular news websites in the US Every hour we download data from to see how many shares, likes and comments has each article, and detect the ones that are rising up the fastest, the ones that are getting viral. facebook Using NLP, we analyze the comments that the facebook users are posting to the viral comments, to get a better understanding what diferentiate the viral posts compared with the average posts. Low level details Using a custom python web parser we crawl, every hour, a list of the top 1000 news websites in the US and add them in a database. The second time we parse the same link, we add it to the database and we calculate the difference in the number of likes, emotions and shares using the facebook API. We for before we stop indexing that particular link. monitor each story 3 days If the story resurface in our databse later, we will monitor again for 3 days before stopping again. Only for around 5% of the posts, the posts that we see that are becoming viral, we download the comments so that we can analyze them and see what people are talking about. The rationale is that we can automatize the process of what is this article about and the validity based on the comments the users post to a article. learning Facebook API payload: # Getting the articles from the FB pages. 1000 FB Pages * 24 hours = 24.000 request per Day. If the average size of the request is 1MB, per DAY need to download 24 GB from Facebook Per Month, this means 720GB # Getting the number of comments, likes, shares from the FB API 1000 news websites * 50 articles per day * 5 days (avg time a article will be crawled) = 250.000 request per hour. Per day this will mean 6.000.000 requests If the average size of the request is 100k, per HOUR i need to download 25 GB from Facebook. Per day 600GB. Per month, minimum bandwidth need of 18TB # Getting the text comments for the Viral Articles. Around 1-5% of all articles. Per Day we will download around 10.000 Viral News/Top News articles Per month this mean 300.000 requests * 100 comment nested pages = If the average size of the request is 1MB, per DAY we need to download 10GB from Facebook Per month, minimum bandwidth needed is 300GB This will be data that we will keep almost entirely, so it`s costly on the server side. 30.000.000 requests Webcrawler API payload: Payload per day of 1000 news websites * 50 articles per day = 50.000 request per DAY. If the average size of the request is 1MB, this means 50GB per day Per month, this means 1.5TB Per month, we need to send over 200M API requests to facebook, downloading over 20TB of data Next steps: I will officialy kickstart this project at the second @Debug Politics Hackaton on the 9th to 11 of December in San Francisco. But i`m working already on it. What i need is somebody with experience in database design, to create the back office architecture. Also, somebody that have experience in Design. I need financing for pushing this project foward, for the server and other costs asociated with the project. If you want to be part in this project, either as a sponsor or as a developer, email me at baditaflorin@gmail.com About Me I collaborate with Rise Project, were i do data analysis and pattern recognition to uncover patterns of data in unstructured datasets. You can find me online on Hackernoon , Medium Florin Badita, AngelList , Twitter , L inkedin, O penstreetmap , G ithub, Q uora, F acebook

Analytica

HackerNoon Mobile

Detecting Fake Viral Stories Before They Become Viral Using FB API

Too Long; Didn't Read

People Mentioned

Companies Mentioned

@baditaflorin

RELATED STORIES