TLDR
Build a Web Crawler with Search bar using Wget and docker-compose [A Step By Step Guide] The solution will be based on: wget crawling a site recursively. tiny PHP script to pass the crawled content from wget to Manticore Search. Tiny php script to make a minimalistic search UI wrapped in docker and docker.compose. The “product” in the end will be barely capable of being used in production for mass web crawling, but if you just need to be able to crawl your own site or a site of your competitor or someone else’s.via the TL;DR App
no story
Written by snikolaev | Database expert. Passionate about databases and search engines.
