I came across the Bright Data MCP (model context protocol) server from Noah Kalson and decided to try my hands on this incredible tool. Noah Kalson Bright Data MCP server is a unified, AI-ready access layer that provides seamless, scalable, and unblockable access to any public websites enabling LLMs, agents and apps to access, discover and extract web data in real-time. Bright Data MCP server The common problem I found worthy of solving was to scrape a public website like Nike.com particularly sneakers for men and have it readily available for users to view showcasing details like the product item, price, availability, and product link. Nike.com So I built this all in one tool that allows for the extraction of its data without any restriction because with Bright Data MCP, your applications can effortlessly retrieve both static and dynamic content from across the web, eliminating the need to build or maintain complex data scraping and unlocking infrastructure. For technologies used in the project, I used: LangChain: allows for the connection to multiple servers or LLM providers at the same time. LangChain: allows for the connection to multiple servers or LLM providers at the same time. LangChain LangGraph: connects to all the request points or nodes together. LangGraph: connects to all the request points or nodes together. LangGraph next.js: A React framework for building scalable and user friendly UI interface. next.js: A React framework for building scalable and user friendly UI interface. next.js Python: for the backend using FastAPI. Python: for the backend using FastAPI. Python Claude (Anthropic API): to orchestrate multiple operation. Claude (Anthropic API): to orchestrate multiple operation. Claude (Anthropic API) Here's a walkthrough of the final product in this demo: CONCLUSION The code is public for viewing, check this repo. this repo Let me know what you think by giving this app a try and searching for your favorite Nike sneaker. this app

Nike

How to Build a Voice Transcription and Translation App with OpenAI Whisper and Streamlit

Portfolio

Nominated for 2022 - HackerNoon Contributor of the Year - Data Visualization

Nominated for 2022 - HackerNoon Contributor of the Year - Heroku

Nominated for 2022 - HackerNoon Contributor of the Year - Javascript

Nominated for 2022 - HackerNoon Contributor of the Year - Frontend

Nominated for 2022 - Remote Work Warrior

Nominated for 2022 - No No No Nodejs

Technical content creator

Too Long; Didn't Read

How to Scrape Any Website Using Bright Data MCP Server and AI Agents

How to Scrape Any Website Using Bright Data MCP Server and AI Agents

About Author

Comments

TOPICS

THIS ARTICLE WAS FEATURED IN

Related Stories

2021: Reviewing and Kaizen-ing My Programming and Writing Life

The Noonification: How Often Do NFTs Pass The Howey Test? (1/13/2023)

Darwin's Hybrid Intelligence to Align AI & Human Goals for Startups & VCs

The Noonification: White Man (11/26/2022)

The Noonification: The Metaverse is a Sh*tshow (11/2/2022)

100 Days of AI Day 1: From Newsletter to Podcast, Leveraging AI for Audio Transformation

2021: Reviewing and Kaizen-ing My Programming and Writing Life

The Noonification: How Often Do NFTs Pass The Howey Test? (1/13/2023)

Darwin's Hybrid Intelligence to Align AI & Human Goals for Startups & VCs

The Noonification: White Man (11/26/2022)

The Noonification: The Metaverse is a Sh*tshow (11/2/2022)

100 Days of AI Day 1: From Newsletter to Podcast, Leveraging AI for Audio Transformation

Light-Mode

Classic

Newspaper

Minty

Dark-Mode

Neon Noir

Minty

HN StartUps