Best match

Most recent

Digi-Key

Volition just raised $11 million to become the same indispensable supplier for mechanical and industrial engineering: think motors, gears, pneumatics, hydraulics, controllers, sensors, measuring and test equipment,

https://www.bing.com/th?id=OVFT.C5YxWLenURpgLJl_YRYuNy&pid=News

TechCrunch

Pittsburgh

Volition wants to be DigiKey for industrial parts

https://www.bing.com/th?id=ODF.M4e-NwwkSCFuKsTp672mxA&pid=news

Get the most recent info and news about Octopart on HackerNoon, where 10k+ technologists publish stories for 4M+ monthly readers.

Engineering tools should be as streamlined and smart as everything else in your life: as your phone, as your email, as all the apps that help you manage your day-to-day. Your work should be as efficient as possible so you can generate innovative ideas, design elegant technology, and manufacture at scale, all because you can focus on the right things. There should not be any distractions caused by poorly designed websites, unreliable data, or unwieldy spreadsheets of haphazard electronic component information. You should be able to compare components, find important technical information, and easily grasp the spectrum of parts that will be a match for your design.

This is what Octopart strives for.

Finding electronic component information should not be holding engineers back from bringing ideas to life quickly and efficiently. We believe that pricing, availability, technical specifications, and reference designs should be transparent, fast to find, and easy to understand. That’s why we create powerful, intelligent tools that streamline the engineering workflow and enable fast decision making. Octopart provides accurate and complete data for millions of electronic components in a way that makes it easy to find and compare parts. Focus on the important stuff: building the technology of the future.

Octopart

Octopath Traveler 2 was released in February 2023 by Square Enix as a sequel to the renowned JRPG Octopath Traveler. Octopath Traveler has been one of the and has received high praise for its amazing visual and sound effects as well as its satisfying battle system and interesting storylines. The only setback that upset many fans was that . most well-received JRPGs of the last decade the storylines of the eight characters weren’t connected in a meaningful way With new stories, features, music, and other aspects introduced, I consider Octopath Traveler 2 an improvement over the amazing previous game. It manages to maintain the high-quality elements of Octopath Traveler while fixing any negative aspects it had by adding more interactions between the eight characters and linking the overall story in a fitting way. In this article, I will review Octopath Traveler 2 on the Nintendo Switch console by examining the story outline, the gameplay, the graphics, and the audio of the game. As a big fan of JRPGs, with some of my latest favorite games being Chained Echoes, Tactics Ogre: Reborn, and -of course- Octopath Traveler, I feel almost obligated to try and review Octopath Traveler 2. Table of Contents: Octopath Traveler 2 Story: Finally the Characters’ Unique Stories are Linked in a More Meaningful Way Plot Characters Octopath Traveler 2 Gameplay: A Vast Detailed World World Exploration Battle Mechanics: An RPG Classic With Some Improvements Octopath Traveler 2 Graphics: A Beautiful and Detailed World Octopath Traveler 2: Soundtrack, Sound Effects, Voice Acting Octopath Traveler 2 Switch Review Verdict Octopath Traveler 2 Story - Finally the Characters’ Unique Stories are Linked in a More Meaningful Way https://www.youtube.com/watch?v=lgmLN1NHXyE&embedable=true Octopath Traveler 2 follows the stories of eight different characters, each with their own unique background. While it is a sequel to Octopath Traveler, it’s not a direct sequel. Instead, Octopath Traveler 2 is a standalone game and you do not need to play Octopath Traveler to fully experience it. Plot The plot is split between eight characters, all of whom decide to leave their hometowns for different reasons. Their adventures bring them together and have them explore the world. Most individual stories manage to maintain immersion from the beginning all the way to the end. For instance, when playing through Osvald’s story or Throne’s story, I was always eager to find out what will happen next and how each scene will play. Other than the story of Agnea, which felt more predictable and less detailed, every other storyline managed to surprise me and keep me on edge in one way or another. I rarely wanted to skip through dialogues, as every bit of information felt important to the interesting plot. As for the ending, it manages to wrap up all eight storylines in an emotional way and serves as the perfect form of closure to an amazing story. However, the game doesn’t explain in much detail how each character decides to join the others at first, and this part feels slightly rushed and vague. Nevertheless, as the stories progress, Travel Banter and Crossed Path stories help everything come together. Compared to the previous game, the individual stories are equally interesting and unique, with the main difference being that they are now linked together in a more meaningful way. Instead of making the game feel like eight different stories, Octopath Traveler 2 tells one well-connected story. Characters The game features eight characters, each with a unique background and story to tell. Here are all of them: - He is the Prince of Ku, a nation that tries to establish peace. However, his warmongering brother takes control of the nation, and Hikari is forced to leave in an attempt to find strong allies who will help him regain control of the nation. Hikari - A merchant who wants to make the world a better place for everyone through the art of trade. Partitio - A thief belonging to an organization named ‘Blacksnakes’ The ‘Blacksnakes’ save children and force them to become thieves (one of the darker storylines). Throne manages to break free and tries to kill the leaders of the ‘Blacksnakes.’ Throne - A wrongfully imprisoned scholar who breaks free and tries to find the man who framed him. Osvald - An inquisitor of the Church of the Sacred Flame who investigates a series of murders against the church and tries to find the truth. Temenos - A local dancer who wants to become a superstar and dance at the most popular venues. Agnea - An apothecary who lost her memories and travels the world in an attempt to regain them. Castti - A beastling who wants to become a hunter and protector of her island. Before becoming a protector, she needs to find three sacred beasts from around the world. Ochette Each character has a unique story full of twists and turns and players get to experience it through a first-person narrative. The best part about the characters of the game is that almost each of them has some unique twist on their story and personality that managed to surprise me. Partitio, despite being a merchant and always looking for a good deal, is anything but greedy; instead, he’d rather spend all his money in an attempt to make the world a better place. Osvald, a renowned scholar who thinks before he acts, becomes blinded by revenge and does everything in his power to find the man who framed him. The most surprising one has been Temenos, who despite being a high-ranked church member, is , even his own beliefs. always ready to question everything Personally, this applies to every character’s story except Agnea. Her storyline didn’t feel unique but predictable. However, every other character managed to surprise me through the unique twists and turns in their stories. Octopath Traveler 2 Gameplay: A Vast Detailed World As for the gameplay, for this Octopath Traveler 2 Switch review, I will focus on the amazing and vast explorable world and the satisfying battle system of the game. Let’s examine them in detail. World Exploration . Each character’s story takes you to a different place with a different theme: from the snowy and unwelcoming tundra of the Winterlands to the vast desert of Hinoeuma, and the dense forests of the Leaflands, the world of Solistia is full of new biomes to explore. The world of Octopath Traveler 2 is vast and full of details The best part is that players are free to explore the world in any order they prefer. They can skip certain places completely or check every corner for hidden items. After all, the game is filled with optional content including sidequests, hidden chests, optional maps, and bosses, as well as hidden secrets. There’s always something new to explore, and the game will reward you for putting in the extra effort. While the stories mentioned in sidequests are less compelling compared to the main storyline, the rewards are well worth it. For instance, progressing side quests, optional maps, and bosses is the only way for players to , which can then be exchanged at the Armsmaster with some of the strongest weapons in the game. get their hands on the Rusty Weapons One minor setback is that, occasionally, players will have to stop exploring around to grind in certain areas. This is because the next quest or area will require a higher level, and even if you’re doing sidequests and exploring around, you’ll still need to grind for a few extra levels. However, this is just a minor bump. The game’s storylines would feel more connected without it, but grinding has always been a part of JRPGs and Octopath Traveler 2 only has a small/normal amount of it. Especially when following the , you won’t have to grind as much. best tips on how to level up fast Battle Mechanics: An RPG Classic With Some Improvements over Octopath Traveler. The combat system is the classic RPG system with a few improvements Each enemy is weak to certain weapons and magic attacks, and players have to identify enemy weaknesses. Each time they hit a weak spot, the enemy takes more damage and loses points of their shield. When the shield is broken, enemies will take more damage from all attacks and lose their next action. Ideally, you always want to break enemies first with weak attacks and then use your strongest moves to damage them while broken. This is ideally paired with the BP (Battle Points) mechanic, where each character gains one battle point on each turn. At any point, they can spend these points to empower their next move. Also, each character is unique in the sense that they have a Latent Power. This charges up during combat while taking damage and breaking enemy shields and offers a unique effect when activated based on the characters. These powers can vary from gaining maximum BP in an instant to dealing heavy AoE damage to enemies. The goal is to fill up your BPs, then break enemies, and hit them with your strongest moves while they’re broken. Overall, the battle system was extremely satisfying and even the fights against common enemies in the open world were interesting. I wanted to learn the weaknesses of every enemy and was always trying to deal more damage compared to the previous fight. Each map included a wide variety of enemies, to the point where I needed to use different characters for each boss. Most bosses were exceptionally challenging but in an interesting way. At first, I went into each boss fight expecting to lose but while trying to find out their weaknesses and attack patterns. Then, I came back better prepared and tried to utilize and experiment with various character combos to deal as much damage as possible based on their weaknesses. Each boss fight felt unique and brought some new mechanics to the game that can be learned only through trial and error. For instance, there is a boss in the optional Decaying Temple mini-dungeon that has three different phases and can one-shot you if you don’t have the right build. https://www.youtube.com/watch?v=-fX4qM2sbXw&embedable=true Octopath Traveler 2 Graphics: A Beautiful and Detailed World In terms of graphics and special effects, Octopath Traveler 2 manages to create a picturesque world full of detail. It features the classic style of RPG games while also adding small details to make the world feel more alive. This includes features like slow-moving cloud shadows, lighting changes through the scenes, and dust hovering while moving through certain areas. The overall image is sharp and full of detail, and Octopath Traveler 2 manages to create a beautiful visual experience. As for performance, the game runs flawlessly on both the Nintendo Switch and the Steam Deck consoles. The game never had any FPS drops and the visuals were stunning. Octopath Traveler 2: Soundtrack, Sound Effects, Voice Acting https://www.youtube.com/watch?v=EpVFw-UyZPA&embedable=true The soundtrack of the game created by Yasunori Nishiki, the composer of the soundtrack in the first game, is phenomenal. Overall, Yasunori Nishiki has received high praise for his work in Octopath Traveler as the . music is tightly connected to the action and doesn’t feel separated The audio always fits the scene and properly sets the mood. It alternates between various themes and helps create immersion. From the heroic tunes playing during battle to the relaxed atmosphere while strolling through the cities, there’s a soundtrack for every scenario. The sound effects are satisfying and make certain actions feel even more rewarding. For instance, breaking an enemy’s shield or using a fully charged skill is always an amazing moment, even if the enemy is only a common foe. Lastly, all major cutscenes from the main storyline are voice acted, which makes the game come alive and tell its story in a great way. The voice actors always managed to deliver the necessary emotion in each scene to make the characters feel alive. I could understand how the characters felt solely based on the voice lines, even without reading the actual dialogue. Octopath Traveler 2 Switch Review Verdict: 9/10 Overall, Octopath Traveler 2 has everything an RPG should have: an immersive storyline with unique characters, a great soundtrack to accompany it, and great graphics and performance to bring everything together. Its only minor setbacks - the extra grind and the vague explanation of how each character joins the party - are overshadowed by all the other amazing elements. Moving on, if Square Enix decides to go for Octopath Traveler 3, the bar will be exceptionally high to the point where it almost feels like a Sisyphean task. However, after finishing Octopath Traveler 2, fans will definitely be asking for a sequel. Have you tried Octopath Traveler 2 yet? What did you think compared to the previous game? Let us know in the comments below. If you’re a fan of RPGs in general and are looking for a new game, consider checking out our list of the . 10 best Mario RPGs ranked by sales

Octopath Traveler 2 Switch Review: An Improvement Over the Previous Game

We live in a data age where data is an important asset in our life and business as well. In the business area, data can be used to collect insights, analyze and make data-driven decisions to solve particular business problems or challenges. Most of the data is collected from websites, apps, social media platforms, and other online platforms. You can use web scraping tools to collect data that you want from the web instead of copying and pasting data manually. The more advanced way is to use software tools, such as Octoparse, which can help you collect data from any website without coding skills within an hour, not in days. What is Octoparse? Octoparse is a visual software tool for anyone who wants to extract different types of data from websites without coding skills. If you don't have coding skills, then Octoparse is the right software tool for you. Octoparse has a lot of features that make it easier and faster for you to get data from the website(s) within a short period. It has a cloud service feature that allows you to run scrapers and access data on the Octoparse Cloud Platform. This is a good service if you plan to collect a large amount of data from the web. Who can use Octoparse? Octoparse can be used by anyone who works on a project where you need to scrape data from the web. Don't worry about having programming skills, you don't have to come from a programming background to use the Octoparse software tool. New Features in Octoparse 8.4 Octoparse 8.4 comes with new features that you don't want to miss. With the brand new version, it gets faster, easier, and more robust to scrape data from the web. 1. New layout The workflow in Octoparse 8.4 is switched from the left to the right side. You can also choose the layout you want (horizontal or vertical). The horizontal view will allow you to view the data row by row whereas, in the vertical view, you can view a specific line and view all data fields vertically. This view is very useful when you want to add a batch of data at the same time. 2. Integration Zapier is now available in Octoparse 8.4, it allows you to integrate Octoparse with the most popular apps like Slack, Google Drive, and many more. With Zapier, you can automate your work and have more time for what matters most. Note: you can learn more and try it . here 3. Backup local data to the Cloud You can now backup the local data you have scrapped to the Octoparse cloud platform. You just need to install the new version of Octoparse 8.4 and start using the new feature. Note: This feature is available for enterprise users only. It is open to users with professional plans as well. How to download and Install Octoparse 8.4 Here are the three steps you need to follow to make sure you have Octoparse 8.4 in your machine. (a) Download the installer Go to website: https://www.octoparse.com/download and download Octoparse 8.4 or click this link . here Note: If you are a Mac user, click and follow the same instruction. here (b) Install Octoparse 8.4 on your machine. Follow the installation instructions to install Octoparse 8.4. Make sure your machine has all requirements necessary to run octoparse 8.4. (c) Log in with your account. After completing installing octoparse 8.4, you can now log in with your account. Sign up if you don't have an account yet. here Use Case: Step by Step on How to Scrap Smartphone Data In this use case, you will learn how to scrap phone data from the website and save them into a CSV file. Let's get started. Step 1: Launch Octoparse The first step is to launch Octoparse in your machine and then log in with your account. Step 2: Enter URL After login, you need to enter the following URL ( ) in the main field and click the start button. https://www.tanzaniatech.one/bei/ The octoparse will load the page and start detecting the data available on the web page. Step 3: Specify Data to Extract When detection is completed, you will see a tip panel that will show you the instruction to follow. Also, you will be able to see data on the preview section We will collect the following details for each phone or laptop listed on the website. Name of the phone/laptop CPU Details RAM details Storage details Display details Camera details OS name URL of the phone Price of the phone Rating Note: Octoparse has already automatically detected all the data attributes we want to collect. You can see the list on the preview panel. Since the phone/laptop details are listed on more than one page, we need to specify the pagination loop to scrape data from the rest of the pages. On the Tips panel, click the edit button as shown below. Here we need to confirm the "Next" button on the page so that octoparse can click on the "Next" button automatically and scrap the data we have specified. If you don't want to extract data from all pages available, you can also specify the number of pages you want to extract data by selecting them on the Tip panel. Note: The "Next" button is detected automatically and highlighted on the web page. Step 4: Run the Task Our next step is to run the task we have specified. You need to click the "Create workflow" button on the Tip panel to save the settings and auto-generate the workflow. Finally, you need to click "Run task now" to start running the task and scrape data from the web pages. Note: You can choose where to run the task either on your device or on the Octoparse cloud (if you are a premium user). Step 5: Export the Data After scraping the data, click the "Export Data" button to save the data on your device. You also have some options to choose the format to save your data.The available options are Excel, CSV,HTML,JSON or export to database. Finally, you can write the name of your file and Octoparse will save the file on your device with the name and file format you have specified. For this example, we can choose to save the file in .xlsx format. Here is the sample of data we have scraped from the website. Now you can analyze and explore the data you have extracted the way you. Conclusion In this article, you have learned how to use Octoparse to scrape data from any website you want and save data on your device. Octoparse is a powerful data extraction tool and it is very easy to use to extract data from complex websites such as amazon.com and others. You can also choose different task templates to extract data from popular websites such as eBay, Facebook, Instagram, Youtube e.t.c If you learned something new or enjoyed reading this article, please share it so that others can see it. Until then, see you in the next article!

How to Get Start No-Code Web Scraping with Octoparse

Revealing the Digital Detective: Essential OSINT Tools and Techniques for Investigators

All scrapers can be split into 2 categories that require different infrastructure and techniques: . Good example is , it wanders around the internet, downloads all the pages it can reach and indexes them in the Google search engine. There are quite a lot of articles and books on how to do generic scraping, so I’m not going to go into details here about it. Generic scrapers Googlebot This kind of scrapers is when you need to download some pages on the website and extract some structured valuable information. Examples of such scrapers are news scrapers, competitors prices analysis, making a local copy of publicly available data (e.g. US patents). If you need this data just once then it’s fine to just write a simple script and run it on you local machine, but if the data needs to be updated regularly then you’ll soon want to make sure it runs reliably and you have observability over current system status. Targeted scrapers. Now let’s try to design a system that can execute targeted scrapers regularly. The system only has one functional requirement: it must be able to execute arbitrary scrapers on schedule. What about non-functional requirements? It must reliably schedule and execute scrapers It should not DDoS the domains being scraped It should be highly available It should properly handle stuck scrapers (infinite loops, lost worker, etc) It should provide monitoring for itself and for the scrapers being executed Here’s an example of how such a system could look like. In result we have following components: stores information about scraper schedule and parameters that are passed to the scraper logic implementation. For example this could be a static configuration file (if you don’t need to reconfigure scrapers on the fly) or standalone database. Scraper configs storage is used to store results and status of the scraper task. Scrapers jobs storage enqueues scraper jobs based on their configs. It needs to make sure that there is no other active job before enqueuing a new one. Scheduler consumes the jobs from the queue and executes the scraper logic. It also keeps job status up-to-date, for example it should mark a job as completed or failed after the logic executing is finished. Worker Now let’s see how everything works together: loads all scrapers configs and starts scheduling the jobs. Scheduler registers a job in with a status Scheduler Scrapers jobs storage Queued enqueues registered job in the Jobs queue Scheduler dequeues a scraper job from the Jobs queue and marks it as Worker Started spawns that executes scraper logic. It also starts a thread which continuously pings that the job is still active so doesn’t enqueue another job Worker Scraper runner Scheduler executes scraper logic. Scraper runner All of the requests made by the scraper logic are intercepted by which can be used for: Requests middleware Rate limiting requests to certain domains Automatically checking if the request is blocked by robots.txt Using proxy Modifying the request (e.g. adding some headers) Logging logic may save its results somewhere or call some APIs based on data it has scraped Scraper When the scraper logic is executed, the marks the job as or . Scraper runner Succeeded Failed Scraper runner How does Scraper runner execute scrapers logic? There are multiple various options. Let’s discuss how we can solve this problem in Python since most scrapers are most commonly written using it. One of the approaches that is used in is to . Basically you package your application using egg format and then Scrapyd spawns a process that loads the provided egg file and executes it. In this case we have several advantages: Scrapyd deploy eggified Python packages Packages may include dependencies in the egg file and different scrapers may use different versions of the same dependency. This can be especially useful if your system is multi-tenant. It’s possible to dynamically deploy scrapers. In this case scraper developers and platform owners are decoupled. Another approach can be used when it’s not required for the scraper developers and the platform owners to be decoupled, e.g. when only one team develops scrapers and maintains the scraping platform. In this case what can be done is to have a scraper runner implementation which imports all the scrapers implementations and then executes them based on what it has dequeued from the queue. Here’s an example implementation: # Scraper implementation interface
class ScraperABC(ABC):
    @property
    @abstractmethod
    def name(self):
        ...

    @abstractmethod
    def execute(config: Any) -> Any:
        ...

# Scraper config that is passed to the Scraper implementation
@dataclass
class ScraperConfig:
    ...

# Sample Scraper implementation
class SampleScraper(ScraperABC):
    @property
    @abstractmethod
    def name(self):
        return "sample_scraper"

    @override
    def execute(config: ScraperConfig) -> Any:
        # <do the actual scrapping>
        text = requests.get("https://example.com").text
        return text[:100]


class Worker:
    def __init__(self, scrapers: list[ScraperABC]):
        self.queue = ...
        self.scrapers = {
          scraper.name: scraper 
          for scraper in scrapers
        }

    # Scraper runner implementation
    def execute_scraper(self, name: str, config: Any) -> Any:
        if name not in self.scrapers:
            raise Exception(f"Unknown scraper {name}")
        return self.scrapers[name].execute(config)

    # Implement consuming the queue
    def start(self):
        while True:
            task = self.queue.dequeue()
            self.execute_scraper(
                task.scraper_name, 
                task.scraper_config
            )

if __name__ == "__main__":
    Worker(
        scrapers=[
            SampleScraper()
        ]
    ).start() This approach is simpler to maintain and has less caveats although it has some limitations: All of the scrapers share dependencies, which may not be a big deal since this platform is not intended to be multi-tenant. Deploying new scrapers is not dynamic and is coupled with deploying the platform which may slow down the release cycle. Availability and Scalability of the platform The availability and scalability of the platform comes down to its components. To have a bit more clear understanding on which components to choose let’s do some back-of-the-envelope calculations. Imagine we have: 1000 scrapers that run every hour Each scraper on average runs for 15 minutes (therefore we’ll have around 250 running scrapers at any point of time) Each scraper needs around 0.1 CPU (they usually do mostly IO so we don’t expect much of CPU usage) and 512 MB of memory (so in total just for running scrapers we’ll need 25 CPU and 125 GB of memory) On average 10 scrapers’ configs are changed within a day - we have mostly read traffic and almost zero write traffic: Scraper configs storage If we use static configuration then there’re no problems at all, we can just distribute the copy across all of the components. We can as well use SQL or NoSQL databases to make the system dynamic. It needs to serve around 16 RPM as each scraper needs to reload scraper config before starting executing it. - the queue only needs to serve around 16 RPM, so we shouldn’t aim for anything heavy: Job Queue We can implement a queue via RDBMS if we have one already Another option is to use distributed queues such as , , , RabbitMQ NATS Redis Kafka - given we have 250 scrapers running at any time and each of them regularly performs health check (so we can kill stuck/timed out tasks), we should have something that can handle 250 write RPS and almost zero read traffic: Scrapers jobs storage Modern RDBMS should handle this traffic without sweating Another option is to use something like Redis, as it most likely ok to not have strong durability guarantees for the scraper jobs. - each scraper job should only be scheduled by one scheduler instance, so how can we achieve this? There are couple of options: Scheduler Have one scheduler replica - if it’s ok to have some downtime for the scheduler it’s a good option since it’s the simplest one. In case we don’t tolerate the downtime we can have primary and secondary scheduler replicas. How can we ensure that secondary replica won’t be active? It can be done via acquiring a distributed lock for the right to schedule jobs. To do so we can use either RDBMS or NoSQL DB that are already used in the system. Last approach is to shard the scheduled scrapers across multiple replicas, but it only makes sense if number of scrapers is really huge and cannot fit into the memory - each worker is a stateless service so we can scale it vertically (by adding more resources) or horizontally (by adding more replicas) Worker In result the final setup can look something like this: RDBMS (e.g. MySQL) used for: Scrapers config storage Job Queue Scrapers jobs storage Scheduler global lock Scheduler + Workers Monitoring There are two parts that needs to be monitored: the platform itself and executed scrapers. Let’s start with the platform monitoring. We want to have an observability over following things: Resource utilisation: CPU, Memory, Disk usage of all the components Queue metrics: Number of pending jobs Processing lag (time since first queue item was enqueued) Average time spent in the queue Enqueue/Dequeue latency Scheduler metrics: Number of active scheduler replicas Difference between expected scheduled time and actual scheduled time (scheduler delay) Worker metrics Number of active workers replicas Difference between expected scheduled time and when the task processing started (end-to-end platform delay) Success rate of the executed tasks Number of timed out tasks All these metrics should tell us how well platform performs overall. As a platform we could also expose some metric for the scrapers developers so they can keep them up-to-date easily. Here are some metrics that come to mind: Task success rate over different periods of time - to tell if the scraper implementation is flaky or completely broken. Number of consecutive failures - this tells us that the scraper is most likely broken at all and should be fixed as soon as possible. Time since last succeeded task - this ensures we execute scrapers within SLAs (e.g. we may have a requirement to provide the data within 1 day of its releasing) Execution time - this tells us if we are able to process the data as soon as it arrives and don’t fall behind (e.g. if getting some information about 1 day period takes 2 days we’ll never be able to scrape the data up to a current point of time, hence the scraper needs to be optimised) Resource consumption (if we spawn a process for each scraper that should be fairly easy to measure) Existing solutions There are actually not that many existing solutions that provide full fledged scraping experience. The most popular is - an open source and collaborative framework for extracting the data from websites. In a fast, simple, yet extensible way. There are multiple ways to host it: Scrapy - if you’d like to host it using your own resources Scrapyd or - cloud solutions to host your spiders Scrapy Cloud ScrapeOps is another popular solution to create and run scrapers that use headless browsers such as or . Although there’s no built-in scheduling and monitoring capabilites so you’ll either need to implement them yourself or use to host the scrapers. Crawlee Playwright Selenium Apify Another notable option is to use no-code solutions like or which allows you to train scrapers by clicking on what you want to scrape. parsehub octoparse

Designing a Scraping Platform: Generic Scrapers vs. Targeted Scrapers

116 Stories To Learn About Web Scraping

So let’s start! Tell us a bit about yourself. For example, name, profession, and personal interests. I’m Raymond Camden. I’m currently employed as a Senior Developer Evangelist for Adobe, mostly focused on our (think doing stuff with PDFs, Office docs, etc at scale) but also involved with our as well. I’ve worked in the dev rel space “professionally” for 10 years or so, and unofficially for about ten before that. I’m mostly focused on the web platform, and dabble in low code/no code solutions, serverless, and enterprise cat demo creation. I’ve run a blog (raymondcamden.com) for twenty years now and have published books on web development. (Some of them even actually sold a few copies.) document APIs Photoshop APIs I’m married to a wife as nerdy as me and we’re parents to eight kids (and no, that’s not a typo). I’m a huge Star Wars fan, but like Star Trek, sci fi, fantasy, as well. Interesting! What was your latest Hackernoon Top story about? My latest is a story on in Alpine. I discovered Alpine about a year ago and fell in love with its simplicity and ease of use. It advertises itself as a “modern jQuery” and that feels completely appropriate for how it works. Previously I was heavy into Vue.js, and while I still have a lot of respect for Vue, Alpine feels more targeted to what I do. updating and supporting URL parameters Do you usually write on similar topics? If not, what do you usually write about? Yes. :) I’ve been working on the web since about 1995 or so. During that time I went from mostly focused on the back end (I spent many years in the ColdFusion community) to mostly front-end topics. I’ve seen the platform grow at an astonishing rate and I’ve loved to see how much better things are now for developers than when I started back in the stone age. Great! What is your usual writing routine like (if you have one?) The most important part of my routine is tracking my ideas. If I don’t write down an idea I’ve had, I’m most definitely going to forget about it. I’ve used various solutions over the years, but lately, I’ve really enjoyed using Microsoft To Do. Generally, I’ll build a demo for my blog post ahead of time, and then write about it, usually in one setting. Being a writer in tech can be a challenge. It’s not often our main role, but an addition to another one. What is the biggest challenge you have when it comes to writing? For me it’s promotion. I track how well my content is going by looking at page views and such, and I do promote my stories, but I could do better. Currently my “routine” is to post on Mastodon, then and . Finally, I republish my work on Hackernoon a week or so later. Twitter LinkedIn What is the next thing you hope to achieve in your career? Honestly, I still feel like I have a decade or more ahead of me in learning about developer relations. I love this field and hope to continue in it till I retire. Wow, that’s admirable. Now, something more casual: What is your guilty pleasure of choice? Listening to trance/techno/pop music at unsafe volumes. I also hold the high score in our family for Ellie Goulding’s “Lights” on Just Dance. Do you have a non-tech-related hobby? If yes, what is it? Books, video games, movies. I’m currently wrapping up the “Shadow and Bone” trilogy and “Octopath 2” for the Switch. Like a lot of folks, I took up baking during Covid, and while I didn’t get deep into it at all, I enjoy it still today. I’m really looking forward to Diablo 4 in a week or two. This is the only game both my wife and I play, so it will be fun to go deep into that for a few months. What can the Hacker Noon community expect to read from you next? I’m not sure. :) I can say I’ve been meaning to look into Microsoft’s mapping solutions (in my last job I was a developer advocate for HERE, a mapping company), and I still want to dig into web components more. What’s your opinion on HackerNoon as a platform for writers? I mainly use it to import stories from my blog, and it works really well for that. Thanks for taking time to join our “ ” series. It was a pleasure. Do you have any closing words? Meet the writer Thanks for having me, and may the force be with you!

Octopart

octopart.com

31 employees

Since 2007

COMPANY RANKING

Octopart

EVERGREEN INDEX #1989

See more notable links about Octopart

Apply to get your Tech Company News Page live on HackerNoon.com today!

Read More Tech Stories Related to #Octopart

Octopart

Octopart

octopart.com

31 employees

Since 2007

COMPANY RANKING

Octopart

EVERGREEN INDEX #1989

Related CompaniesRank

Octopart Story MentionsComplete List

See more notable links about Octopart

Apply to get your Tech Company News Page live on HackerNoon.com today!

Read More Tech Stories Related to #Octopart

Octopart