Justworks

justworks

Effective Leadership Series Isaac Oates is the Founder and CEO of Justworks, the fastest growing HR technology company that helps entrepreneurs and businesses grow with confidence by giving them access to big-company benefits, automated payroll, HR tools, and compliance support — all in one place. FirstMark is a proud investor in Justworks. Early in his career, Isaac worked concurrently at Amazon and in the United States Army as an intelligence officer—in both roles honing various aspects of his leadership approach. After Etsy acquired his first startup Adtuitive, Isaac led Etsy’s HR and payments group. He founded Justworks in 2012. We recently caught up with him to hear more about his growth as a leader, how he handles delegation as CEO, and much more. What was your first management role? What did you learn from it? I started my career as a software engineer at Amazon, where I worked on “third party payments” (3PP), the system that moved money between buyers and sellers on their marketplace. About three years in, I took on a project management role within the team. Even though no one reported to me on paper, I was in a position to lead the broader team to a successful outcome. That was the first time it really clicked that leadership isn’t about the boxes and lines on an org chart, but about following people that you trust and respect. Early in your career, before you assumed leadership roles, what was one quality you appreciated in a great manager? My first manager at Amazon, Phil Wilson, really cared about me as a person. I was 22, fresh out of school and had just started as an engineer on the payments team. Phil would talk with me about his own career experiences and help me think through the challenges I was facing. I think the big lesson was that he could lead by asking questions rather than telling you what to do. He was really good at letting you draw your own conclusions but also getting you where he wanted. This is something I wish I was better at. It requires a lot of patience. He was much more like a coach or older brother in the support he gave and we’re still in touch to this day. The big lesson was that he could lead by asking questions rather than telling you what to do. What are the most important daily or weekly habits that you’ve developed as a leader? Whether it’s on paper, on a spreadsheet, or in my inbox, sorting tasks into different groups and then attacking them together head-on has helped me be more productive. Recently, for example, I set up some filters in my email account that separate messages that demand a response from everything else. That might seem small, but it’s changes in habit like this that have helped me be much more responsive to the people I work with. It’s made a surprisingly large difference in our operating tempo. I also take time on Sundays to draft emails to send out on Monday morning. It helps me catch up on my inbox without also dumping work on my team over the weekend. What framework do you use for your one-on-one meetings? I like to review the goals that we’ve set together, discuss if any changes are appropriate, and then exchange feedback. I have 30 minutes per month with each of my direct reports, so most of our substantive discussion happens in group meetings rather than 1:1. That’s intentional — I think group meetings are helpful in building shared consciousness and minimizing politics. How do you handle delegation? What and when do you choose to delegate and how’d you become comfortable with it? I’ve been fortunate to hire and work with some extremely talented people at Justworks, who are much better at most things than I am. That makes it easy to delegate. That said, if I’m still getting comfortable with their work style or it’s a high-risk project then I ask to stay involved as it progresses. Some things I’ll always stay closely involved in, for example, activities that impact or send messages about our culture like promotions to leadership roles and company-wide events. If I’m still getting comfortable with their work style or it’s a high-risk project then I ask to stay involved as it progresses. What advice would you give to a first-time manager about giving effective feedback? Giving effective feedback is difficult but it gets easier with time. I don’t have a specific framework, but I do acknowledge the person’s accomplishments and the effort they have put in. I also make it clear that my goal is to help them and the entire company perform better and that they should take the feedback in that vein, rather than criticism just for the sake of it.Giving feedback is probably the #1 management challenge for many people, including very senior and experienced leaders. Just keep working on it. What’s one unique hiring tip you’d share with a first-time manager? Start with the basics. Figure out where the person is from, what they’re about, what they value. Then ask yourself the right questions. Will this person strengthen your team? Do you think they have the potential to be promoted a few times? Do they have the curiosity to keep growing? Sometimes you get lucky and the right person falls in your lap, but usually, it takes a lot of nos to get to a yes. It’s worth taking the time. Want more? Check out our other Effective Leadership interviews with seasoned leaders from , , , and . Box Slack Oscar Trello Is there someone who you really admire as a leader (not at your current company) and would like to see us interview? Send their name and short description to jack@firstmarkcap.com.

Justworks’ Isaac Oates on How To Delegate as CEO + More

When Apple piggybacks on Google, it ‘just works’ better (Image from ) Flickr The other day, my iMac crashed. As it was a 2011 model, I was prepared. Well, sort of. I had been regularly backing up my stuff via Time Machine but had stopped doing this 24/7 a couple of years ago so as to extend my Time Machine’s life. It’s a venerable 2008 model. Also, a lot has changed since 2008 as most of my files are saved online and I don’t need regular backups any more. However when I ran the ‘restore utility’ on my Mac, I realised that I had lost track of time. My Time Machine informed me that my last backup was over four months ago. I mulled it over as my mouse hovered over the ‘restore’ button. By my reckoning, the only new files of any value on my Mac that I added in that period were a few photos and songs. So I gave Time Machine the go ahead with the ‘restore.’ Once the restore was done, the Mac seemed fine. But I could not say the same of apps like Photos, iTunes, Contacts and more. Besides the last four months’ missing files, Photos had also converted many earlier videos to thumbnails. These wouldn’t play when I clicked on them as the original files were missing. iTunes too had randomly made quite a few of my song collections disappear. The Bermuda Triangle To be honest, I wasn’t surprised as it’s not the first time Apple apps have played this trick on me, and it’s not just me. Here another victim the same tune on the Apple Support forum for iTunes. And here’s a about disappearing pictures from the Photos app. The latter seems to have had a happy ending for a few individuals who recovered their photos by tweaking some settings. I’m sure expert users will be able to relocate the missing photos but my heart goes out to the ordinary folks who have lost their precious memories forever. singing thread This sort of thing should not happen, but unfortunately it does with Apple. After several unsuccessful attempts to figure out why, I gave up and concluded there’s a sort of Bermuda Triangle in Appleland where songs and photos and the rest mysteriously disappear. But unlike the Bermuda Triangle, you can’t avoid using apps like Photos and iTunes if you are in the Apple ecosystem. There were two solutions. The first was to make a clean break an emigrate out of the Apple ecosystem. The second was to find a backup system where I am not expected to expect the unexpected, so to say. When Apple just works I had tried the ‘migration’ solution by moving to an Android for a while. But it didn’t work out. I don’t know if this was because I was just more comfortable with Apple, having been on the platform for ages. Or because there were situations when my Android let me down, while Apple actually ‘just worked.’ Let me illustrate with an example. I once had to make a screen recording as a demo of how to use a certain app on Android, and send it to a WhatsApp Group. After making the demo, I sent it from my Android phone to my iPhone via WhatsApp to test if it was working. It wasn’t. Whatever compression WhatsApp had used on the one minute long video had caused the audio and video to go off sync. Now I could troubleshoot and try to figure out what when wrong. But sometimes you just want to get things done instead of spending time figuring out why it doesn’t work. So I followed my gut feeling that the video might work if I sent it from my iPhone. Accordingly, I bluetoothed the video from my Android to my iPhone, and then Whatsapped it again. This time, the video worked perfectly. I don’t know why the problem happened. It could have been an issue with the WhatsApp on Android compressing videos using a method that’s different from WhatsApp on iOS. Or it could have been a hardware issue with my Android phone, a Redmi Note 4 with 4GB RAM and 64GB storage (which are pretty good specs). Or it could have been a problem with Android itself. What I do know was Apple just worked. In that sense, Apple is perfect for people who would rather spend time ‘doing things’ rather than ‘fixing things.’ I guess that’s why the Apple will always have a strong base of loyal users. However my problem is the situation when Apple ‘just works’ is no longer true. When Apple just doesn’t work Since the migrating to Android route wasn’t working for me, I had to check the reliable online backup route. Only one entity fit the bill. Google. One thing I can vouch for after ten years of using Google apps is that there are no Bermuda Triangles. Google is almost a synonym for reliable. Google’s biggest shortcoming is a loss of privacy. Not to mention all those persevering ads trying to sell me a plethora of hair loss remedies, completely oblivious of the fact that I have a full head of hair. A lot of people I know absolutely refuse to have anything to do with Google for this reason. However having , I figured it was worthwhile for me to to share my data with Google in return for free use of their services. This is probably because I don’t have much to hide. Which means I have more to gain and less to lose in sharing my data with Google. To illustrate, Google’s bots once helped me avoid missing a flight by warning me it was due to leave at 5 am while I was under the impression it was leaving at 5 pm. Google was able to do this because I let them read my mails, and signed in to Google’s notifications. But like I said, sharing data with Google is not for everyone. weighed the pros and cons Photos In fact, Google had already solved my photo backup issue a couple of years ago when it offered unlimited online backup of all my photos in the ‘Google Photos’ app. It’s not in full resolution admittedly, but it’s good enough for me. In fact, it even backs up ‘live’ photos as tiny videos. The problem is the ISPs in India used to have strict restrictions on internet bandwidth and download limits (wifi), and speeds were very slow. But things have improved in recent times. My connection has gone from a miserly 30GB/month @4Mbps two years ago, to an acceptable 300GB @60Mbps now. Cost too has gone down to ₹1250 per month ($20). What clinched the deal is Google offers ‘image’ search. Like I can search for ‘elephant,’ and ‘Google Photos’ will visually identify all the elephants in my photo collection. I found this fascinating enough to merit its . In fact, nowadays if I need to find a photo in a hurry, my go-to app on my iPhone is not Apple’s ‘Photos’ but ‘Google Photos.’ Yup, it just works. own post Setting up automatic photo backup in Google is easy. You first install the ‘Google Photos’ app, and give the app permission to access your ‘Photos’ app. Once the app is up and running, click on the icon with 3- horizontal lines on the app’s top left. This will reveal the settings gear icon. Click on that, then tap on ‘backup and sync,’ and toggle on the switch in the next screen. That’s it. Backing up all my photos on Google Photos Music My Music backup experience on Google was a bit odd. Apple Music and Google Play Music had launched in India with free trial offers, with Google also offering a free backup of all my music (probably to compete with Apple Match). I tried out both the services, and also backed up all my music on Google using the Chrome browser on my Mac (see below). As more free music streaming services came out in India, I discontinued my Apple music subscription, and switched to the free service of Google Play Music. Not surprisingly, it’s supported by ads, which isn’t really a deal breaker as there’s an option to skip the longer video ads similar to the YouTube thingy. Uploading my music to Google Play Music via Chrome Some time later, my favorite Nobel Prize winning singer disappeared from my iTunes. Maddening, no doubt, but I sort of expected it by now from the whimsical iTunes. Out of habit, I checked my Google Play Music library, and was surprised to find my entire uploaded music collection including the missing Dylan album, still available online. I do occasionally buy music from Google so my guess is they see me as a paying customer and let me keep my stuff for free in return. Anyway, as I started downloading my lost music, Bob started singing, “If not for you, My sky would fall.” That captured exactly what I felt for Google at that instant. Contacts Apple’s mysterious disappearing tricks also happens with people in my Contacts app. My plumber once vanished into thin air, a disappearing act that drove me nuts enough to spew out an about it. Again, it was Google to the rescue as I had backed up my contacts. entire post Apple actually makes it easy to back up your contacts with Google on your iPhone. Basically you sync your phone’s contacts with your Gmail contact list. This way, you can always retrieve a contact from your ‘google contacts’ if it goes missing from your iPhone. It’s again simple to set up. First, add your Gmail account to your phone’s Mail app. Then go to Settings and enable contacts syncing. As you can see below, my personal gmail account is set to have contacts to be synced on my iPhone. Backing up my contacts on Google The other advantage of this system is you can keep your work contacts out of your phone to prevent excess clutter in your phone book. If you look at the middle image above, you can see my Work email account doesn’t have contacts synced. I save work contacts directly in my work email phonebook, and leave them there. If I need to have a work contact on my phone, I temporarily put him in my phone book. This does make him to appear in my personal Gmail contacts. But once I’m done with the job, I remove the contact from my phone, and it automatically diasappears from my personal Gmail. Should I ever need his number, I can always locate his details from my work email account. There is a better way to prevent clutter in your phonebook. Group all your work contacts in a group called Work, and then hide the group from within the Contacts app. Anytime you wish to see your work contacts, just tap on the groups link on the top left of your Contact app, and toggle the tick mark back on. This is far more convenient method except for one little issue. You can see groups in the Contacts app but you can’t create Groups in it. You have to go to your browser, sign in to your iCloud, and create the Group there. That’s just a bit too much for me. Why not in the app? Apple may have its reasons but they aren’t saying. However my issues with Apple are not just limited to backup. Image Editing Apple’s Photos offers a lot of options in editing images. But Google’s Snapseed is at altogether another level. Let me explain with one feature. Lots of times, when you shoot someone, you get a good mood shot but the details of the face can’t be seen. Now in Photos, you can brighten the whole image and see details. But doing that causes the rest of the picture to get washed out and you lose the mood. With Snapseed, you can easily brighten one particular area, say the face without brightening the rest of the image and washing out the picture. Saving the Mood: My original image VS whole picture brightened (Photos); and just the face brightened (Snapseed) See above the difference between how Photos and Snapseed fixed the image. What I want to do is see the Buddha’s smile so as to create a contrast with the mysterious jungle with menacing creepers circling around his head. With Photos, you can now see the smile on the Buddha’s face after tweaking the image but there is no more mystery as the dark, creepy jungle effect is gone. On the other hand, Snapseed lets me brighten just the Buddha’s face without losing the mood of the rest of the shot. Of course, this is no big deal for those who use editing software like Photoshop. (In fact, it may be possible to do this within Photos for all I know.) But for layman like me to whom Photoshop is a puzzle, this is a picture saver. In case, you want to know how I did it, here’s a screen recording on my iPhone. Using Snapseed to selectively brighten a small part of an image Basically you open the image from within Snapseed, tap on tools, then tap on ‘Selective’ in the second row, then tap the Buddha’s face, and you will see a ‘B’ appear on his face. Now tap the screen with two fingers apart, and a circle will appear. Slowly pinch your two fingers together till the circle only covers the Buddha’s face (the brightening effect only appears inside the circle). Then touch the screen outside and above the image, and slowly swipe right till the brightness count reads 100. To confirm, tap on the window like icon on the top left which gives me a before-after look. Finally, tap the tick mark at the right bottom corner, and then on ‘export’ which send the reworked image back to my Photos app. Image Scanning Another little gem of an app from Google is for which the closest equivalent on iOS is the scanning feature within the iOS Notes app. But unlike that, it’s perfect for scanning old paper photo albums and uploading it to Google Photos. Photos are usually reflective and it’s hard to scan them with a phone camera. PhotoScan gets around this by ingeniously shooting four shots from four corners of your image, and then combining the four shots to give a perfect scan without any reflections. In effect you are digitising your real Photo albums without having to go through the tedious process of using an actual scanner to scan each image. PhotoScan Maps Apple Maps just doesn’t work in India. It’s literally crippled and can’t give directions to any location, and instead says, ‘Directions are not available between these locations.’ I think Google Maps has too much of a headstart that Apple just stopped trying to catch up. In fact, Google is so clued into India that it offers offline maps. This is ideal for a country where the mobile network is non-existent in lots of places, and mobile data is limited. Apple Maps doesn’t work in India, while Google Maps does As for the real-time location sharing on Google Maps with one touch, I think it’s originally an Apple Maps feature. Since Apple Maps is semi-dysfunctional in India, Google gratefully appropriated this unique iPhone 3D Touch feature for Google Maps. Just hard press on the Google Maps icon, and a ‘Send My Location’ option pops up. It’s perfect for when you are driving, and have only a moment to tell someone where you are. YouTube If I wish to share large videos shot on my iPhone, Apple suggests we use the ‘Shared Albums’ feature of Photos. Unfortunately, this uses iCloud whose free version only offers 5GB of space. This 5GB has to be shared for all photos, documents and all my iDevice backups, which means it usually has very little free space. Apple would like me to buy extra space on iCloud. But that’s not going to happen when Google offers me unlimited space for free on YouTube, the world’s favorite video platform. Sharing videos using YouTube That’s why I upload videos shot on my iPhone to YouTube by default. It’s almost as easy to do as iCloud. Basically you go to Photos where your video is saved, select it, tap the share button at the bottom, and then choose YouTube from the share options. You can even ensure your video’s privacy by choosing ‘Private,’ which means only those you share its link with can view it. Mail Apple does give me a free iCloud email, but it’s not half as good as Gmail. So like almost everyone else on Planet Earth, I rely on a Gmail account. For everyday use, I operate my Gmail via the stock Mail app as it’s simple to set up. But Mail is limited in its capabilities, especially search. If the Mail app can’t find something, I usually fire up Google’s Gmail app which easily finds what I am looking for. I know there are many other good email apps. But sometimes having too many email apps to choose from is more stressful than whatever email issues I have. So I stick with combination of Mail and Gmail apps. Apple’s Mail app search feature falls short when compared to the Gmail app Google Drive and Docs Unlike the previous apps, I use these in tandem with Apple’s apps. iCloud is perfect for backing up my phone, and works seamlessly with online docs like Pages and Numbers. But I also use Google Docs as it’s easier to share across platforms. Space on iCloud is also an issue as I mentioned earlier. Google Drive is that way a far more convenient online vault as it allows me to share large files without having to worry about running out of space. Of course, there are other custom apps for large file transfers like Dropbox and WeTransfer. But that often leads to “Now where on earth did I put that file?” situations, which I would rather avoid. iCloud works seamlessly with Apple but limited space makes Google Drive a dependable standby News Google is not always the best alternative when an Apple app falls short. News is one instance where neither Apple nor Google really stand out from the other news apps. Others apps like Flipboard are more popular, at least here in India. Apple News vs Google Feed vs Flipkart — there’s no clear winner There are many more ways in which Google is deeply woven into Apple and improves my whole iPhone experience . Like every time I type something on my iPhone, I do it via , Google’s amazingly accurate swipe-to-type keyboard (it’s also a mini Google, a GIF maker and much more). To get something translated into another languages, I use . To know how to pronounce a word, I type define: xyz in my Safari browser. To play videos from my phone on my TV, I use the app (you need Google’s Chromecast device). To transfer money, I go to . To know if a flight is on time, I just feed in the Flight Number in Google… Gboard Google Translate Google Home Tez One last question. Why don’t I just get the Pixel 2XL, Google’s challenger to the iPhone? Maybe I should. Or maybe I’m just plain greedy and want the best of the two worlds of Apple and Google. That may explain things as I belong to the ‘mixed breeds are healthier than purebreds’ school of thought. iPhoogle - arguably the world’s best phone In short, I love my iPhone, but I love it better when it’s running on Google.

Apple ‘just works’ better when it piggybacks on Google

Let’s talk about something that’s been driving me nuts for years: Google’s ongoing struggles with handling canonical URLs and duplicate content. It’s a problem that, despite being fundamental to search engine optimization (SEO), Google still can’t seem to get right. And while Google continues to flounder, guess who’s quietly acing it? Bing. Yes, you read that right – Bing! It’s honestly baffling. Here we are, in 2024, and Google, the self-proclaimed king of search, can’t even get the basics of web indexing straight. Let’s break this down. The Canonical URL Catastrophe A canonical URL is meant to tell search engines which version of a page is the “official” one when you have multiple URLs with similar or identical content. Seems simple enough, right? You’d think Google, with all its technological might and years of search engine dominance, would have perfected this by now. But nope. Google’s handling of canonical tags is, more often than not, a dumpster fire. Why? Because Google has a tendency to ignore your carefully placed canonical tags, or worse, it just gets them wrong. It might pick the wrong version of your page, or even worse – it might randomly declare some other URL as the canonical without any explanation. This leads to pages getting improperly indexed, with duplicate content spread across multiple URLs and absolutely no way for you to control it. Result? Your rankings plummet, your SEO strategy crumbles, and your site’s traffic tanks – all thanks to Google’s refusal to play by the canonical rulebook. But, oh, here’s where it gets good. Bing? Bing doesn’t have these problems. Bing can crawl your site, see your canonical tag, and follow it like a well-behaved child. It doesn’t second-guess you, doesn’t ignore your instructions, and doesn’t make arbitrary decisions that hurt your visibility. Bing understands that when a site owner points to a canonical URL, that’s what it should respect. It’s as if Bing actually listens to website owners, while Google seems to be playing SEO bingo with your site’s content. Duplicate Content Woes Then there’s duplicate content. Another area where Google’s “genius” just falls flat. We all know duplicate content is a big issue when it comes to SEO. When search engines find multiple copies of the same content, they’re supposed to figure out which one is the most relevant and display it in search results. Seems simple, right? Well, it’s not. Google has been notorious for misidentifying duplicate content, sometimes even penalizing sites with pages that have only slight variations. I’ve seen it happen time and again: you publish a well-researched article, and somehow, Google finds another page on your site and decides they’re too similar, tanking your rankings. Why does this happen? It’s because Google’s algorithm, despite being incredibly complex, still can’t reliably handle duplicate content in certain situations. There are cases where Google just can’t seem to distinguish between genuinely different pages and what’s essentially the same content presented slightly differently – even if those pages are clearly differentiated by their canonical tags or intended purpose. The result? Duplicate content penalties, frustrated webmasters, and a whole lot of wasted time trying to get Google to "understand" your site. Now, Bing? Bing doesn’t overcomplicate things. It doesn’t see minor content variations and freak out like Google does. It doesn’t penalize you for every slight difference between two pages that have the same general topic. Instead, Bing takes a more balanced approach to identifying and handling duplicate content. You won’t find yourself scrambling to fix duplicate content issues that Google invented in the first place. It just works. The Moral of the Story Why is Bing, a search engine that many still dismiss, consistently getting the basics right while Google is out here making life unnecessarily difficult for website owners? The answer is simple: Google is overcomplicating things, and it’s failing to acknowledge the importance of consistency and accuracy in indexing and content management. In contrast, Bing seems to have a more grounded, practical approach. Bing might not dominate the search engine market the way Google does, but at least when it comes to handling canonical URLs and duplicate content, Bing is getting it right. It’s time for Google to stop pretending that its “complex algorithms” are the answer to everything. Sometimes, simplicity works better. Maybe it’s time for Google to take a page out of Bing’s playbook and fix the very issues that have been plaguing SEO professionals for years. Because if Bing can get it right, there’s absolutely no excuse for Google to keep getting it wrong. Let’s talk about something that’s been driving me nuts for years: Google’s ongoing struggles with handling canonical URLs and duplicate content. It’s a problem that, despite being fundamental to search engine optimization (SEO), Google still can’t seem to get right. And while Google continues to flounder, guess who’s quietly acing it? Bing. Yes, you read that right – Bing! And while Google continues to flounder, guess who’s quietly acing it? Bing. Yes, you read that right – Bing! It’s honestly baffling. Here we are, in 2024, and Google, the self-proclaimed king of search, can’t even get the basics of web indexing straight. Let’s break this down. The Canonical URL Catastrophe A canonical URL is meant to tell search engines which version of a page is the “official” one when you have multiple URLs with similar or identical content. Seems simple enough, right? You’d think Google, with all its technological might and years of search engine dominance, would have perfected this by now. But nope. Google’s handling of canonical tags is, more often than not, a dumpster fire. Why? Because Google has a tendency to ignore your carefully placed canonical tags, or worse, it just gets them wrong. It might pick the wrong version of your page, or even worse – it might randomly declare some other URL as the canonical without any explanation. ignore randomly This leads to pages getting improperly indexed, with duplicate content spread across multiple URLs and absolutely no way for you to control it. Result? Your rankings plummet, your SEO strategy crumbles, and your site’s traffic tanks – all thanks to Google’s refusal to play by the canonical rulebook. SEO strategy But, oh, here’s where it gets good. Bing ? Bing doesn’t have these problems. Bing can crawl your site, see your canonical tag, and follow it like a well-behaved child. It doesn’t second-guess you, doesn’t ignore your instructions, and doesn’t make arbitrary decisions that hurt your visibility. Bing Bing understands that when a site owner points to a canonical URL, that’s what it should respect. It’s as if Bing actually listens to website owners, while Google seems to be playing SEO bingo with your site’s content. Duplicate Content Woes Then there’s duplicate content. Another area where Google’s “genius” just falls flat. We all know duplicate content is a big issue when it comes to SEO. When search engines find multiple copies of the same content, they’re supposed to figure out which one is the most relevant and display it in search results. Seems simple, right? Well, it’s not. Google has been notorious for misidentifying duplicate content, sometimes even penalizing sites with pages that have only slight variations. I’ve seen it happen time and again: you publish a well-researched article, and somehow, Google finds another page on your site and decides they’re too similar, tanking your rankings. Why does this happen? It’s because Google’s algorithm, despite being incredibly complex, still can’t reliably handle duplicate content in certain situations. There are cases where Google just can’t seem to distinguish between genuinely different pages and what’s essentially the same content presented slightly differently – even if those pages are clearly differentiated by their canonical tags or intended purpose. The result? Duplicate content penalties, frustrated webmasters, and a whole lot of wasted time trying to get Google to "understand" your site. Now, Bing? Bing doesn’t overcomplicate things. It doesn’t see minor content variations and freak out like Google does. It doesn’t penalize you for every slight difference between two pages that have the same general topic. Instead, Bing takes a more balanced approach to identifying and handling duplicate content. You won’t find yourself scrambling to fix duplicate content issues that Google invented in the first place. It just works. The Moral of the Story Why is Bing, a search engine that many still dismiss, consistently getting the basics right while Google is out here making life unnecessarily difficult for website owners? The answer is simple: Google is overcomplicating things, and it’s failing to acknowledge the importance of consistency and accuracy in indexing and content management . Google is overcomplicating things, and it’s failing to acknowledge the importance of consistency and accuracy in indexing and content management In contrast, Bing seems to have a more grounded, practical approach. Bing might not dominate the search engine market the way Google does, but at least when it comes to handling canonical URLs and duplicate content, Bing is getting it right. It’s time for Google to stop pretending that its “complex algorithms” are the answer to everything. Sometimes, simplicity works better. Maybe it’s time for Google to take a page out of Bing’s playbook and fix the very issues that have been plaguing SEO professionals for years. Because if Bing can get it right, there’s absolutely no excuse for Google to keep getting it wrong.

Why Bing Doesn’t Struggle with Canonical URLs and Duplicate Content, But Google Does!

I've been thinking about starting a new project using my go-to tech: Clojure. For me, it's the best tool around, and I’m always surprised that more people don’t see its potential. It’s a fantastic multi-purpose language design to build solid software, so I asked myself: What’s a good fit for it where I feel confident building something meaningful in my spare time? After some tinkering, I come up with this idea: A workflow automation tool. Think Zapier, but designed specifically for small and medium-sized businesses that need reliable and observable workflows. I believe Clojure’s strengths—its "everything is a map" data model and its powerful concurrency support—make it the perfect language for building robust API integrations. Why Build Another Workflow Tool? There are plenty of workflow automation tools like Zapier that offer a huge range of integrations. I can't challenge these platforms by being cheaper or offering more integrations. So, I started thinking: Why don’t I use these tools more often in my day-to-day work? I think it’s because of reliability, observability, and missing collaboration features. These platforms often feel like black boxes—when something goes wrong, it's hard to know why. Collaboration is often an afterthought: you need the business++ plan and the UX for it has been shoved into the product — hard to use something like that if you're in an organization larger than a startup. Worse for me, workflows are not easily observable: you're left hoping everything works as expected. In today’s world, there is no reason to guess whether your software is functioning. With the right tech, you can observe and ensure that everything works as expected. Building Software That “Just Works” I’m a big believer in building software that delivers value from day one. No overselling. No bugs lurking beneath the surface. Too many SaaS promise a lot but deliver a half-baked experience — as soon as you get out of the main use case of the tool, you have to contact the support to figure out what is going on. Nobody wants to spend time on a half-bake tool selling you promises. Users want value now, not after your next fundraising. My Initial Target I’m focusing on the basics: Create and edit workflows: The UX to configure the tool.
View workflow execution: View what works and what does not
Webhook integration: The basic block to build on top off
User roles and permissions: Enable teams to work together seamlessly
Export and edit workflows programmatically: Building block that allows integration to developer tools
Workflow status via API: Clear visibility into the state of every workflow.
On-demand pricing: Pay for what you use, when you use it. Once these core functionalities are in place, I’ll start adding more integrations—things like email, AWS APIs, tools for deployment, Notion (I use it a lot), Slack, human interaction workflows, and Google Workspace. Over time, I’ll figure out what other integrations make sense for users who share my vision of a workflow platform. Adding LLM support seems obvious in today’s market. Every company now needs to integrate with LLMs, managing things like throttling, token usage, and cost monitoring. I see real value in handling that complexity for users. Of course, I could be wrong—maybe companies will prefer to build their own LLM integrations. That reminds me of the beginning of the mobile apps business, I thought building mobile apps would be a flop because companies would have to maintain applications for search platforms and still do a responsive website to convince users to install their application. I was dead wrong, and now most companies maintain two mobile applications and at least one website, and eventually two when they have a website for mobile. The Challenge of Selling How will I market this project? Honestly, I’m not sure yet. My initial plan is to create something useful for myself that’s solid enough for others to use. Whether I manage to find users is still a big question mark, but I know I need to figure it out and articulate my value proposition clearly. Why I Will Succeed I’m confident in my ability to build this. I know how to leverage infrastructure to create something that’s both solid and scalable. With Clojure’s strengths and my experience, I believe I can deliver something valuable. Why I Could Fail Marketing and sales: This is my weakest area. I have limited experience, and finding the first users will be tricky. I also don’t enjoy platforms like Twitter, and this kind of social website with a tidbit of knowledge and a lot of noise. I prefer long-form content, but it’s harder to reach people that way and build an audience. I need to not underestimate the issue.
Market misjudgment: It’s possible that the workflow automation market won’t take off the way I expect. Companies will keep building integration after integration and discover over time that it is a costly handover to build and *maintain (*emphasis on maintaining as software estimation only accounts for the cost of building things and not on the cost of observing, debugging, upgrading, interconnecting, …)
Time constraints: Balancing family, a day job and other interests means I might not be able to dedicate as much time to this project as I’d like. Why This Will Make Me Happy Ultimately, it’s about the journey. This project will push me out of my comfort zone and challenge me to learn new things, not just in my area of expertise, but across the full software lifecycle. I’m excited to tackle the problems I’m less familiar with and grow from these experiences, regardless of the final outcome. I've been thinking about starting a new project using my go-to tech: Clojure. For me, it's the best tool around, and I’m always surprised that more people don’t see its potential. It’s a fantastic multi-purpose language design to build solid software, so I asked myself: What’s a good fit for it where I feel confident building something meaningful in my spare time? What’s a good fit for it where I feel confident building something meaningful in my spare time? After some tinkering, I come up with this idea: A workflow automation tool. Think Zapier, but designed specifically for small and medium-sized businesses that need reliable and observable workflows. I believe Clojure’s strengths—its "everything is a map" data model and its powerful concurrency support—make it the perfect language for building robust API integrations. Why Build Another Workflow Tool? There are plenty of workflow automation tools like Zapier that offer a huge range of integrations. I can't challenge these platforms by being cheaper or offering more integrations. So, I started thinking: Why don’t I use these tools more often in my day-to-day work? Why don’t I use these tools more often in my day-to-day work? I think it’s because of reliability, observability, and missing collaboration features. These platforms often feel like black boxes—when something goes wrong, it's hard to know why. Collaboration is often an afterthought: you need the business++ plan and the UX for it has been shoved into the product — hard to use something like that if you're in an organization larger than a startup. Worse for me, workflows are not easily observable: you're left hoping everything works as expected. In today’s world, there is no reason to guess whether your software is functioning. With the right tech, you can observe and ensure that everything works as expected. Building Software That “Just Works” I’m a big believer in building software that delivers value from day one. No overselling. No bugs lurking beneath the surface. Too many SaaS promise a lot but deliver a half-baked experience — as soon as you get out of the main use case of the tool, you have to contact the support to figure out what is going on. Nobody wants to spend time on a half-bake tool selling you promises. Users want value now, not after your next fundraising. My Initial Target I’m focusing on the basics: Create and edit workflows: The UX to configure the tool. View workflow execution: View what works and what does not Webhook integration: The basic block to build on top off User roles and permissions: Enable teams to work together seamlessly Export and edit workflows programmatically: Building block that allows integration to developer tools Workflow status via API: Clear visibility into the state of every workflow. On-demand pricing: Pay for what you use, when you use it. Create and edit workflows : The UX to configure the tool. Create and edit workflows View workflow execution: View what works and what does not View workflow execution: Webhook integration : The basic block to build on top off Webhook integration User roles and permissions : Enable teams to work together seamlessly User roles and permissions Export and edit workflows programmatically : Building block that allows integration to developer tools Export and edit workflows programmatically Workflow status via API : Clear visibility into the state of every workflow. Workflow status via API On-demand pricing : Pay for what you use, when you use it. On-demand pricing Once these core functionalities are in place, I’ll start adding more integrations—things like email, AWS APIs, tools for deployment, Notion (I use it a lot), Slack, human interaction workflows, and Google Workspace. Over time, I’ll figure out what other integrations make sense for users who share my vision of a workflow platform. Adding LLM support seems obvious in today’s market. Every company now needs to integrate with LLMs, managing things like throttling, token usage, and cost monitoring. I see real value in handling that complexity for users. Of course, I could be wrong—maybe companies will prefer to build their own LLM integrations. That reminds me of the beginning of the mobile apps business, I thought building mobile apps would be a flop because companies would have to maintain applications for search platforms and still do a responsive website to convince users to install their application. I was dead wrong, and now most companies maintain two mobile applications and at least one website, and eventually two when they have a website for mobile. The Challenge of Selling How will I market this project? Honestly, I’m not sure yet. My initial plan is to create something useful for myself that’s solid enough for others to use. Whether I manage to find users is still a big question mark, but I know I need to figure it out and articulate my value proposition clearly. Why I Will Succeed I’m confident in my ability to build this. I know how to leverage infrastructure to create something that’s both solid and scalable. With Clojure’s strengths and my experience, I believe I can deliver something valuable. Why I Could Fail Marketing and sales: This is my weakest area. I have limited experience, and finding the first users will be tricky. I also don’t enjoy platforms like Twitter, and this kind of social website with a tidbit of knowledge and a lot of noise. I prefer long-form content, but it’s harder to reach people that way and build an audience. I need to not underestimate the issue. Market misjudgment: It’s possible that the workflow automation market won’t take off the way I expect. Companies will keep building integration after integration and discover over time that it is a costly handover to build and *maintain (*emphasis on maintaining as software estimation only accounts for the cost of building things and not on the cost of observing, debugging, upgrading, interconnecting, …) Time constraints: Balancing family, a day job and other interests means I might not be able to dedicate as much time to this project as I’d like. Marketing and sales : This is my weakest area. I have limited experience, and finding the first users will be tricky. I also don’t enjoy platforms like Twitter, and this kind of social website with a tidbit of knowledge and a lot of noise. I prefer long-form content, but it’s harder to reach people that way and build an audience. I need to not underestimate the issue. Marketing and sales Market misjudgment : It’s possible that the workflow automation market won’t take off the way I expect. Companies will keep building integration after integration and discover over time that it is a costly handover to build and *maintain (*emphasis on maintaining as software estimation only accounts for the cost of building things and not on the cost of observing, debugging, upgrading, interconnecting, …) Market misjudgment Time constraints : Balancing family, a day job and other interests means I might not be able to dedicate as much time to this project as I’d like. Time constraints Why This Will Make Me Happy Ultimately, it’s about the journey. This project will push me out of my comfort zone and challenge me to learn new things, not just in my area of expertise, but across the full software lifecycle. I’m excited to tackle the problems I’m less familiar with and grow from these experiences, regardless of the final outcome.

Starting a New Side Project: Yet Another Workflow Tool

Cross-Platform Design Wrapped Part 3: UI Polish and Interaction Design

Cross-Platform Design Wraped Deel 3: UI Pools en Interaksie Ontwerp

የፕላትፎርም ንድፍ የታሸገ ክፍል 3፡ UI ፖላንድኛ እና መስተጋብር ንድፍ

Cross-Platform Design Wrapped Del 3: UI Polish og Interaction Design

Diseño multiplataforma: Parte 3: Mejora de la interfaz de usuario y diseño de interacción

クロスプラットフォーム デザインのまとめ パート 3: UI の洗練とインタラクション デザイン

Kelių platformų dizaino įvyniota 3 dalis: UI poliravimas ir sąveikos dizainas

Cross-Platform Design Wrapped Deel 3: UI Polish en Interaction Design

Кроссплатформенный дизайн в обертке, часть 3: UI-полировка и дизайн взаимодействия

Naqshadaynta Madal-isku-tallaabta ee La Duubay Qaybta 3: UI Polish iyo Naqshad Is-dhexgalka

Дизајн на више платформи, део 3: Пољски кориснички интерфејс и дизајн интеракције

Cross-Platform Design na Nakabalot sa Bahagi 3: UI Polish at Interaction Design

Thiết kế đa nền tảng được gói gọn trong Phần 3: Đánh bóng giao diện người dùng và thiết kế tương tác

Users getting locked out of their own accounts is an all-too-common scenario. After just a few typos, they can no longer try again until time passes or they reset their passwords with an email. As frustrating as it is, at least it stops hackers — or does it? Statistics suggest otherwise. Over one-third of all Americans have had their social media accounts hacked despite standard attempt limits. Why don’t these protections stop cybercriminals if they can lock out the users themselves, and how can people stay safe? How Account Lockouts Are Supposed to Stop Hackers Account lockouts are supposed to stop a type of hack known as a “brute-force” attack. At its simplest, brute forcing involves trying a string of random inputs until something works. More often than not, cybercriminals use automated tools to do this, which are much faster than manually guessing passwords. The idea behind login attempt limits is that getting a password right will take far more than three or so guesses. Consequently, locking the account after so many attempts theoretically stops brute-force attacks before they succeed. However, things rarely play out this way. How Hackers Get Around Account Lockouts Cybercriminals can get into a password-protected account in several ways. Here are a few strategies they use to get past account lockouts, even in a brute-force attack. Offline Brute-Force Attacks Account lockouts would work if hackers attempted to guess a password on the login screen. The problem is that they don’t often do that. Instead, they perform offline brute-force attacks, where they steal password data and try to break through it in a different environment where there are no attempt limits. Attackers cannot be locked out after multiple failed attempts when they possess the raw encrypted data. That’s because they’re not trying to decrypt it online, where the server has these protections. Rather, they take just the account data and brute force it on their own computer or on a different, unsecured server. These attacks require stealing the passwords from a website first, then using brute force tools to break through the encryption. While that’s more complicated than simply guessing credentials on-site, it gives criminals time. Even if it takes millions of attempts, they can reveal the password in a few days and then login like a normal user on the legitimate site. Unfortunately, it often doesn’t need to take millions of attempts. Despite years of warnings from security experts, “password” is still the most common base term used in passwords, and 18% of passwords contain only lowercase letters. Offline brute-force attacks are often made easier for hackers simply because users don’t follow password length and complexity best practices. Credential Stuffing Another option is to use credential stuffing. Here, hackers take login info they know worked for one account and use it to get into another. They often get those credentials from past data breaches, where other cybercriminals have sold stolen usernames and passwords on the dark web. Just 12% of global internet users always use new credentials when opening a new account. Most people use the same passwords across multiple sites — sometimes all of them. As a result, it’s a safe bet that a stolen passcode will work somewhere else, so hackers can use one to log into an account in just one or two attempts. Social Engineering Hackers can also work around account lockouts through social engineering. This is such a broad category of attacks, so it can cover several strategies to steal or bypass login credentials. The most direct way is to trick users into telling attackers their passwords by posing as a trusted source. Alternatively, cybercriminals may send an email claiming to be from a legitimate site with a link to log into their account. However, the link leads to a fraudulent login page identical to the real one where criminals can see what users type in. Such attacks may seem obvious, but 298,878 people fell for phishing attempts in 2023 alone. That’s more than any other form of cybercrime and suggests social engineering is still highly effective. Keylogging and Man-in-the-Middle Attacks Another way attackers can avoid account lockouts is by watching users as they type in passwords. There are two main approaches here — keylogging software and man-in-the-middle (MITM) attacks. Keyloggers are a form of malware cybercriminals may deliver through phishing, malicious websites, or other means. Once installed, they track what users type, including passwords, which hackers can use to log into people’s accounts in a single attempt. MITM attacks are similar but involve intercepting users’ inputs — which can include passwords — before they reach the server. Encryption can stop these attacks, but public Wi-Fi or unsecured websites are susceptible to them. How Can Users Stay Safe? It’s safe to say account lockouts are not enough to stop hackers. Thankfully, users can protect themselves by following a few other best practices. Better safety starts with using stronger passwords. Experts recommend using random password generators, as these create more complex strings of characters that are harder to brute force. Never reusing passwords and periodically changing them is also a good idea. These steps will make credential stuffing less effective. Users should also enable multifactor authentication (MFA) wherever available. It’s still possible to brute force past MFA, but it’s much harder than getting past a simple username-password combo. Stopping Cybercriminals Is Far From Simple Brute force attacks are not as simple as they seem at first, and defending against them is rarely straightforward. While an account lockout system makes sense in theory, it is not safe enough to be the sole defense in practice. Cybercriminals have many tools at their disposal, so strong protections likewise use multiple ways of staying safe. Pairing login limits with long, complex, and unique passwords, MFA, and frequent credential changes will offer the most security. Users getting locked out of their own accounts is an all-too-common scenario. After just a few typos, they can no longer try again until time passes or they reset their passwords with an email. As frustrating as it is, at least it stops hackers — or does it? Users getting locked out of their own accounts is an all-too-common scenario. After just a few typos, they can no longer try again until time passes or they reset their passwords with an email. As frustrating as it is, at least it stops hackers — or does it? Statistics suggest otherwise. Over one-third of all Americans have had their social media accounts hacked despite standard attempt limits. Why don’t these protections stop cybercriminals if they can lock out the users themselves, and how can people stay safe? one-third of all Americans one-third of all Americans How Account Lockouts Are Supposed to Stop Hackers Account lockouts are supposed to stop a type of hack known as a “brute-force” attack. At its simplest, brute forcing involves trying a string of random inputs until something works. More often than not, cybercriminals use automated tools to do this, which are much faster than manually guessing passwords. The idea behind login attempt limits is that getting a password right will take far more than three or so guesses. Consequently, locking the account after so many attempts theoretically stops brute-force attacks before they succeed. However, things rarely play out this way. How Hackers Get Around Account Lockouts Cybercriminals can get into a password-protected account in several ways. Here are a few strategies they use to get past account lockouts, even in a brute-force attack. Offline Brute-Force Attacks Account lockouts would work if hackers attempted to guess a password on the login screen. The problem is that they don’t often do that. Instead, they perform offline brute-force attacks, where they steal password data and try to break through it in a different environment where there are no attempt limits. Attackers cannot be locked out after multiple failed attempts when they possess the raw encrypted data. That’s because they’re not trying to decrypt it online, where the server has these protections. Rather, they take just the account data and brute force it on their own computer or on a different, unsecured server. cannot be locked out cannot be locked out These attacks require stealing the passwords from a website first, then using brute force tools to break through the encryption. While that’s more complicated than simply guessing credentials on-site, it gives criminals time. Even if it takes millions of attempts, they can reveal the password in a few days and then login like a normal user on the legitimate site. Unfortunately, it often doesn’t need to take millions of attempts. Despite years of warnings from security experts, “password” is still the most common base term used in passwords, and 18% of passwords contain only lowercase letters. Offline brute-force attacks are often made easier for hackers simply because users don’t follow password length and complexity best practices. the most common base term the most common base term Credential Stuffing Another option is to use credential stuffing. Here, hackers take login info they know worked for one account and use it to get into another. They often get those credentials from past data breaches, where other cybercriminals have sold stolen usernames and passwords on the dark web. Just 12% of global internet users always use new credentials when opening a new account. Most people use the same passwords across multiple sites — sometimes all of them. As a result, it’s a safe bet that a stolen passcode will work somewhere else, so hackers can use one to log into an account in just one or two attempts. 12% of global internet users 12% of global internet users Social Engineering Hackers can also work around account lockouts through social engineering. This is such a broad category of attacks, so it can cover several strategies to steal or bypass login credentials. The most direct way is to trick users into telling attackers their passwords by posing as a trusted source. Alternatively, cybercriminals may send an email claiming to be from a legitimate site with a link to log into their account. However, the link leads to a fraudulent login page identical to the real one where criminals can see what users type in. Such attacks may seem obvious, but 298,878 people fell for phishing attempts in 2023 alone. That’s more than any other form of cybercrime and suggests social engineering is still highly effective. 298,878 people fell for phishing attempts 298,878 people fell for phishing attempts Keylogging and Man-in-the-Middle Attacks Another way attackers can avoid account lockouts is by watching users as they type in passwords. There are two main approaches here — keylogging software and man-in-the-middle (MITM) attacks. Keyloggers are a form of malware cybercriminals may deliver through phishing, malicious websites, or other means. Once installed, they track what users type, including passwords, which hackers can use to log into people’s accounts in a single attempt. MITM attacks are similar but involve intercepting users’ inputs — which can include passwords — before they reach the server. Encryption can stop these attacks, but public Wi-Fi or unsecured websites are susceptible to them. How Can Users Stay Safe? It’s safe to say account lockouts are not enough to stop hackers. Thankfully, users can protect themselves by following a few other best practices. Better safety starts with using stronger passwords. Experts recommend using random password generators , as these create more complex strings of characters that are harder to brute force. recommend using random password generators recommend using random password generators Never reusing passwords and periodically changing them is also a good idea. These steps will make credential stuffing less effective. Users should also enable multifactor authentication (MFA) wherever available. It’s still possible to brute force past MFA , but it’s much harder than getting past a simple username-password combo. possible to brute force past MFA possible to brute force past MFA Stopping Cybercriminals Is Far From Simple Brute force attacks are not as simple as they seem at first, and defending against them is rarely straightforward. While an account lockout system makes sense in theory, it is not safe enough to be the sole defense in practice. Cybercriminals have many tools at their disposal, so strong protections likewise use multiple ways of staying safe. Pairing login limits with long, complex, and unique passwords, MFA, and frequent credential changes will offer the most security.

Why Hackers Aren't Stopped by Account Lockouts

Proč hackery nezastaví uzamčení účtů

Hvorfor hackere ikke stoppes af kontolåse

Por qué los bloqueos de cuentas no detienen a los piratas informáticos

מדוע האקרים לא נעצרים על ידי נעילת חשבונות

アカウントロックアウトでハッカーを阻止できない理由

Неліктен хакерлер тіркелгіні құлыптау арқылы тоқтатылмайды

Pourquoi ba Hackers Ba Arrêter Te Na Ba Lockouts Ya Compte

Kāpēc kontu bloķēšana neaptur hakerus

Dlaczego hakerzy nie są zatrzymywani przez blokady kont

ولې هیکرز د اکاونټ لاک آوټ لخوا نه بندیږي

Prečo hackerov nezastaví blokovanie účtov

Зашто хакере не заустављају блокаде налога

Why You Don't Need PNPM And YARN

কেন আপনার পিএনপিএম এবং সুতা লাগবে না

Warum Sie PNPM und YARN nicht benötigen

Por qué no necesita PNPM ni HILO

Pourquoi vous n'avez pas besoin de PNPM et de YARN

आपको PNPM और YARN की आवश्यकता क्यों नहीं है?

PNPM と YARN が必要ない理由

PNPM과 YARN이 필요하지 않은 이유

Por que você não precisa de PNPM e YARN

Почему вам не нужны PNPM и YARN

Neden PNPM ve İPLİK'e İhtiyacınız Yok

Tại sao bạn không cần PNPM và YARN

为什么你不需要 PNPM 和 YARN

In this exclusive interview, Josh shares his journey into blockchain, discusses the challenges and opportunities facing QuickNode, and offers valuable insights into the latest trends in blockchain technology. From the potential of rollups to advice for emerging talent, Josh provides a comprehensive look at the current state and future direction of blockchain infrastructure. Join us as we delve into the world of blockchain development, exploring how QuickNode is working to make Web3 more accessible and efficient for developers and businesses alike. Ishan Pandey: Hello Josh Neuroth, It's a pleasure to welcome you to our 'Behind the Startup' series. Can you share your journey into the blockchain space and what drew you to QuickNode? Josh Neuroth: I’ve spent most of my career working on internet infrastructure, scaling networks, clouds, and data centers for leading technology companies. Web3 seemed the natural evolution: decentralized computing, allowing anyone anywhere to build applications and communities on chain! I started working on web3 stuff in 2019 and went full-time in early 2021. I joined QuickNode in 2023 because it was best positioned to help onboard traditional businesses into web3. The first place builders look at is getting access to the blockchain —which is QuickNode’s core product. Ishan Pandey: With your extensive experience in scaling infrastructure, what do you see as the biggest challenges and opportunities for QuickNode in the current blockchain ecosystem? Josh Neuroth: The amount of blockchains has exploded.. and now with rollup infrastructure, anyone can launch their own blockchain. One opportunity at QuickNode we’re focusing on is building the best place for businesses to build applications on blockchain tech. Developers need reliable infrastructure that just works. The biggest challenge currently facing us and the rest of the space is the lack of a regulatory framework. While we offer infrastructure and cloud services which are well understood by regulators, our customers need regulatory certainty. Ishan Pandey: How does QuickNode’s advanced infrastructure differentiate it from other blockchain node providers in the market? Josh Neuroth: QuickNode differentiates itself through its reliable, scalable, and user-friendly infrastructure. Our platform offers top-tier RPC access, node hosting, and API services that ensure high performance and reliability. What sets us apart is our focus on developer experience. We provide comprehensive tools like faucet services, marketplace add-ons, and advanced analytics, simplifying the development and deployment process. Additionally, our support for a wide range of blockchain protocols and our ability to handle high transaction volumes make us a preferred choice for enterprise customers and smaller development teams. QuickNode’s commitment to continuous innovation and customer satisfaction further strengthens our position in the market. Ishan Pandey: Can you elaborate on the benefits of QuickNode’s Rollups-as-a-Service platform for developers, particularly in terms of flexibility and performance? Josh Neuroth: QuickNode’s Rollups-as-a-Service (RaaS) platform offers significant benefits in terms of flexibility and performance. For developers, our RaaS platform provides essential tools such as high-quality RPC access, sequencer support, faucet services, and marketplace add-ons. These tools eliminate the need for extensive infrastructure management, allowing developers to focus on building and scaling their applications. The flexibility of our platform enables developers to customize and deploy rollups according to their specific needs, while our robust infrastructure ensures high performance and reliability. By handling the complexity of setting up and maintaining nodes, QuickNode significantly reduces development time and costs, helping developers accelerate their project timelines and bring their innovations to market faster. Ishan Pandey: What new trends in technology are you particularly interested in, and how is your business leveraging these trends to enhance its services? Josh Neuroth: I’m particularly interested in rollups, because it offers a new use case for web3 and the overall crypto ecosystem. Blockchain rollups —building a blockchain on top of another chain like Ethereum —  allow organizations and communities to monetize their block space by offering their chain as a form of serverless compute. In effect, it represents a new paradigm for computing. Think of a rollup as several technologies in one: Cloudflare Workers, Stripe Payments, and Reddit all in one. I think this opens many doors… imagine building an app for a gaming community in the Philippines and not having to think about credit card payments. Ishan Pandey: What advice do you have for up-and-coming talent in the blockchain space, especially those looking to innovate in blockchain infrastructure and Web3 services? Josh Neuroth: My advice for up-and-coming talent is to try to build something —even if it's just a side project. The blockchain industry is rapidly evolving, and staying updated with the latest developments and technologies is crucial. Use the tech, experiment, and try to understand what’s happening behind the interface. Many of my last hires stood out because they built something. Some of the projects were just proof of concepts. I’ve looked through thousands of resumes, and every person who built something got an interview. Don’t forget to like and share the story! Vested Interest Disclosure: This author is an independent contributor publishing via our business blogging program. HackerNoon has reviewed the report for quality, but the claims herein belong to the author. #DYOR. In this exclusive interview, Josh shares his journey into blockchain, discusses the challenges and opportunities facing QuickNode, and offers valuable insights into the latest trends in blockchain technology. From the potential of rollups to advice for emerging talent, Josh provides a comprehensive look at the current state and future direction of blockchain infrastructure. QuickNode, Join us as we delve into the world of blockchain development, exploring how QuickNode is working to make Web3 more accessible and efficient for developers and businesses alike. Ishan Pandey: Hello Josh Neuroth, It's a pleasure to welcome you to our 'Behind the Startup' series. Can you share your journey into the blockchain space and what drew you to QuickNode? Ishan Pandey: Hello Josh Neuroth, It's a pleasure to welcome you to our 'Behind the Startup' series. Can you share your journey into the blockchain space and what drew you to QuickNode? Ishan Pandey: Hello Josh Neuroth, It's a pleasure to welcome you to our 'Behind the Startup' series. Can you share your journey into the blockchain space and what drew you to QuickNode? Josh Neuroth: I’ve spent most of my career working on internet infrastructure, scaling networks, clouds, and data centers for leading technology companies. Web3 seemed the natural evolution: decentralized computing, allowing anyone anywhere to build applications and communities on chain! I started working on web3 stuff in 2019 and went full-time in early 2021. I joined QuickNode in 2023 because it was best positioned to help onboard traditional businesses into web3. The first place builders look at is getting access to the blockchain —which is QuickNode’s core product. Josh Neuroth: Ishan Pandey: With your extensive experience in scaling infrastructure, what do you see as the biggest challenges and opportunities for QuickNode in the current blockchain ecosystem? Ishan Pandey: With your extensive experience in scaling infrastructure, what do you see as the biggest challenges and opportunities for QuickNode in the current blockchain ecosystem? Ishan Pandey: With your extensive experience in scaling infrastructure, what do you see as the biggest challenges and opportunities for QuickNode in the current blockchain ecosystem? Josh Neuroth: The amount of blockchains has exploded.. and now with rollup infrastructure, anyone can launch their own blockchain. One opportunity at QuickNode we’re focusing on is building the best place for businesses to build applications on blockchain tech. Developers need reliable infrastructure that just works. The biggest challenge currently facing us and the rest of the space is the lack of a regulatory framework. While we offer infrastructure and cloud services which are well understood by regulators, our customers need regulatory certainty. Ishan Pandey: How does QuickNode’s advanced infrastructure differentiate it from other blockchain node providers in the market? Ishan Pandey: How does QuickNode’s advanced infrastructure differentiate it from other blockchain node providers in the market? Ishan Pandey: How does QuickNode’s advanced infrastructure differentiate it from other blockchain node providers in the market? Josh Neuroth: QuickNode differentiates itself through its reliable, scalable, and user-friendly infrastructure. Our platform offers top-tier RPC access, node hosting, and API services that ensure high performance and reliability. What sets us apart is our focus on developer experience. We provide comprehensive tools like faucet services, marketplace add-ons, and advanced analytics, simplifying the development and deployment process. Additionally, our support for a wide range of blockchain protocols and our ability to handle high transaction volumes make us a preferred choice for enterprise customers and smaller development teams. QuickNode’s commitment to continuous innovation and customer satisfaction further strengthens our position in the market. Josh Neuroth: Ishan Pandey: Can you elaborate on the benefits of QuickNode’s Rollups-as-a-Service platform for developers, particularly in terms of flexibility and performance? Ishan Pandey: Can you elaborate on the benefits of QuickNode’s Rollups-as-a-Service platform for developers, particularly in terms of flexibility and performance? Ishan Pandey: Can you elaborate on the benefits of QuickNode’s Rollups-as-a-Service platform for developers, particularly in terms of flexibility and performance? Josh Neuroth: QuickNode’s Rollups-as-a-Service (RaaS) platform offers significant benefits in terms of flexibility and performance. For developers, our RaaS platform provides essential tools such as high-quality RPC access, sequencer support, faucet services, and marketplace add-ons. These tools eliminate the need for extensive infrastructure management, allowing developers to focus on building and scaling their applications. The flexibility of our platform enables developers to customize and deploy rollups according to their specific needs, while our robust infrastructure ensures high performance and reliability. By handling the complexity of setting up and maintaining nodes, QuickNode significantly reduces development time and costs, helping developers accelerate their project timelines and bring their innovations to market faster. Josh Neuroth: Ishan Pandey: What new trends in technology are you particularly interested in, and how is your business leveraging these trends to enhance its services? Ishan Pandey: What new trends in technology are you particularly interested in, and how is your business leveraging these trends to enhance its services? Ishan Pandey: What new trends in technology are you particularly interested in, and how is your business leveraging these trends to enhance its services? Josh Neuroth: I’m particularly interested in rollups, because it offers a new use case for web3 and the overall crypto ecosystem. Blockchain rollups —building a blockchain on top of another chain like Ethereum —  allow organizations and communities to monetize their block space by offering their chain as a form of serverless compute. In effect, it represents a new paradigm for computing. Think of a rollup as several technologies in one: Cloudflare Workers, Stripe Payments, and Reddit all in one. I think this opens many doors… imagine building an app for a gaming community in the Philippines and not having to think about credit card payments. Josh Neuroth: Ishan Pandey: What advice do you have for up-and-coming talent in the blockchain space, especially those looking to innovate in blockchain infrastructure and Web3 services? Ishan Pandey: What advice do you have for up-and-coming talent in the blockchain space, especially those looking to innovate in blockchain infrastructure and Web3 services? Ishan Pandey: What advice do you have for up-and-coming talent in the blockchain space, especially those looking to innovate in blockchain infrastructure and Web3 services? Josh Neuroth: My advice for up-and-coming talent is to try to build something —even if it's just a side project. The blockchain industry is rapidly evolving, and staying updated with the latest developments and technologies is crucial. Use the tech, experiment, and try to understand what’s happening behind the interface. Many of my last hires stood out because they built something. Some of the projects were just proof of concepts. I’ve looked through thousands of resumes, and every person who built something got an interview. Josh Neuroth: Don’t forget to like and share the story! Vested Interest Disclosure: This author is an independent contributor publishing via our business blogging program. HackerNoon has reviewed the report for quality, but the claims herein belong to the author. #DYOR. Vested Interest Disclosure: This author is an independent contributor publishing via our business blogging program . HackerNoon has reviewed the report for quality, but the claims herein belong to the author. #DYOR. Vested Interest Disclosure: business blogging program business blogging program

QuickNode VP of Product Josh Neuroth on Blockchain Infrastructure, Rollups, and Web3 Innovation

ব্লকচেইন পরিকাঠামো, রোলআপস এবং ওয়েব3 উদ্ভাবনে প্রোডাক্ট জোশ নিউরোথের কুইকনোড ভিপি

Josh Neuroth, VP of Product bei QuickNode, über Blockchain-Infrastruktur, Rollups und Web3-Innovation

Josh Neuroth, vicepresidente de producto de QuickNode, sobre infraestructura Blockchain, paquetes acumulativos e innovación Web3

Josh Neuroth, vice-président du produit QuickNode, sur l'infrastructure Blockchain, les rollups et l'innovation Web3

क्विकनोड के उत्पाद उपाध्यक्ष जोश न्यूरोथ ने ब्लॉकचेन इंफ्रास्ट्रक्चर, रोलअप और वेब3 इनोवेशन पर बात की

QuickNode のプロダクト担当副社長 Josh Neuroth がブロックチェーン インフラストラクチャ、ロールアップ、Web3 イノベーションについて語る

블록체인 인프라, 롤업 및 Web3 혁신 부문 Josh Neuroth 제품 부문 QuickNode 부사장

Vice-presidente de produto da QuickNode, Josh Neuroth, sobre infraestrutura Blockchain, rollups e inovação Web3

Вице-президент QuickNode по продуктам Джош Нейрот по инфраструктуре блокчейна, накопительным пакетам и инновациям Web3

QuickNode Ürün Başkan Yardımcısı Josh Neuroth'un Blockchain Altyapısı, Toplamalar ve Web3 İnovasyonu

Phó chủ tịch sản phẩm QuickNode Josh Neuroth về Cơ sở hạ tầng Blockchain, Bản tổng hợp và Đổi mới Web3

QuickNode 产品副总裁 Josh Neuroth 谈区块链基础设施、Rollups 和 Web3 创新

The Address Resolution Protocol’s (ARP) primary function is to resolve, or map, Internet Protocol (IP) version 4 addresses to Media Access Control (MAC) addresses enabling devices on the same Local Area Network (LAN) to discover each other and communicate at the data link layer (Layer 2) of the Open Systems Interconnection (OSI) model as defined by RFC 826. How ARP Works Finding a Device: When your device wants to talk to another device on the network but doesn’t know its MAC address, it sends out an ARP broadcast request. This request asks, “Who has this IP address?”


Getting a Response: Every device on the network gets this request. The device with the matching IP address replies with its MAC address.


Communication: Now, your device knows the MAC address of the device it wants to talk to and can send data directly to it. In environments where devices frequently join and leave the network or change their IP addresses (e.g., dynamic IP addresses assigned by DHCP) permanently storing these IP/MAC pairings could lead to an invalid network state potentially causing connectivity issues or delays. Also, in the case of laptops that connect to multiple networks, you would need to manually set the IP/MAC pairings, so the magical (it just works) dynamic nature is mostly advantageous. To speed up communication avoiding multiple repetitions of ARP broadcasts, operating systems employ ARP caching that stores the IP/MAC information temporarily, or in other words, gives it a lifetime. Normally this lifetime is a few minutes, but this value is configurable, although requiring some technical ability. A low lifetime ensures that the ARP cache is regularly updated to reflect the current network state. Also, with a low ARP cache lifetime, malicious ARP entries introduced by an attacker are removed more quickly, limiting the window of opportunity for attacks like ARP spoofing or poisoning. This increases the difficulty for attackers to disrupt your device communication. Noteworthy is that static ARP entries are immune to spoofing or poisoning, but with the overhead of manual intervention when IP changes need to occur or there’s a Network Interface Card (NIC) that needs to be replaced (as the MAC will be different). When you are competing, you won’t be changing your NIC, and the network state is not expected to change either (unless a new gamer joins and your machine speaks to theirs directly), so it’s advantageous to use static ARP entries to speed up communication and reduce network traffic. Also, ensuring that anyone on the same subnet as you isn’t able to poison your ARP cache for competitive advantage is healthy and, allow me to say it: smart! What the criminal could do (it is a criminal offense), is to poison your ARP cache so that some communication would not be sent to the right device but to a non-existing one; you would probably perceive it as latency (the time it takes for data to travel from one point to another in a network) or some other problem, giving them an advantage.) I won’t share how to disrupt your adversaries and not be easily caught any further than what I already did. So, don’t even ask me, as we won’t be friends. Just know that without the need to regularly verify the IP and MAC address of the router, access point, or switch your competitive device is connected to, you’ll be able to squeeze a few milliseconds every couple of minutes while maintaining a stronger security posture. And to go even a step further, ensuring that no other ARP traffic along with any other not required traffic is able to be sent to your competitive machine, as your operative system firewall will block it, you may just dodge any network misshapes or less ethical adversaries (in fact criminals, it’s not fun, it’s a crime). How to Speed Up and Protect Your Device? You need to know the MAC/IP address pairing of the router, switch, and access point that your device is connected to; 2- You need to set a static ARP entry for it; 3- You need to enforce robust firewall rules that will allow you to focus all traffic on the task at hand. How to do the above? Well, first, I hope you appreciate and enjoy that I’ve already shared a secret (give the article some love in exchange will you?), so don’t expect me to share everything as perhaps one day we may compete against each other. And unless system tuning is against the rules, I intended to make use of it. Besides, I’m quite sure you can figure it out… if you are a seriously competitive person. If all of the above is a bit above your head and you are worried you may become a victim of a criminal during competition, enquire with the organizer of the games if they will have dynamic ARP inspection devices to protect the players. If they do not, point them to this article. The Address Resolution Protocol’s ( ARP ) primary function is to resolve, or map, Internet Protocol ( IP ) version 4 addresses to Media Access Control ( MAC ) addresses enabling devices on the same Local Area Network ( LAN ) to discover each other and communicate at the data link layer (Layer 2) of the Open Systems Interconnection ( OSI ) model as defined by RFC 826 . ARP IP MAC LAN OSI RFC 826 How ARP Works Finding a Device: When your device wants to talk to another device on the network but doesn’t know its MAC address, it sends out an ARP broadcast request. This request asks, “Who has this IP address?” Getting a Response: Every device on the network gets this request. The device with the matching IP address replies with its MAC address. Communication: Now, your device knows the MAC address of the device it wants to talk to and can send data directly to it. Finding a Device: When your device wants to talk to another device on the network but doesn’t know its MAC address, it sends out an ARP broadcast request. This request asks, “Who has this IP address?” Finding a Device : When your device wants to talk to another device on the network but doesn’t know its MAC address, it sends out an ARP broadcast request. This request asks, “Who has this IP address?” Finding a Device Getting a Response: Every device on the network gets this request. The device with the matching IP address replies with its MAC address. Getting a Response : Every device on the network gets this request. The device with the matching IP address replies with its MAC address. Getting a Response Communication: Now, your device knows the MAC address of the device it wants to talk to and can send data directly to it. Communication : Now, your device knows the MAC address of the device it wants to talk to and can send data directly to it. Communication In environments where devices frequently join and leave the network or change their IP addresses (e.g., dynamic IP addresses assigned by DHCP ) permanently storing these IP/MAC pairings could lead to an invalid network state potentially causing connectivity issues or delays. DHCP Also, in the case of laptops that connect to multiple networks, you would need to manually set the IP/MAC pairings, so the magical (it just works) dynamic nature is mostly advantageous. To speed up communication avoiding multiple repetitions of ARP broadcasts, operating systems employ ARP caching that stores the IP/MAC information temporarily, or in other words, gives it a lifetime. Normally this lifetime is a few minutes, but this value is configurable, although requiring some technical ability. A low lifetime ensures that the ARP cache is regularly updated to reflect the current network state. Also, with a low ARP cache lifetime, malicious ARP entries introduced by an attacker are removed more quickly, limiting the window of opportunity for attacks like ARP spoofing or poisoning. This increases the difficulty for attackers to disrupt your device communication. Noteworthy is that static ARP entries are immune to spoofing or poisoning, but with the overhead of manual intervention when IP changes need to occur or there’s a Network Interface Card (NIC) that needs to be replaced (as the MAC will be different). When you are competing, you won’t be changing your NIC, and the network state is not expected to change either (unless a new gamer joins and your machine speaks to theirs directly), so it’s advantageous to use static ARP entries to speed up communication and reduce network traffic. Also, ensuring that anyone on the same subnet as you isn’t able to poison your ARP cache for competitive advantage is healthy and, allow me to say it: smart! What the criminal could do (it is a criminal offense), is to poison your ARP cache so that some communication would not be sent to the right device but to a non-existing one; you would probably perceive it as latency (the time it takes for data to travel from one point to another in a network) or some other problem, giving them an advantage.) I won’t share how to disrupt your adversaries and not be easily caught any further than what I already did. So, don’t even ask me, as we won’t be friends. Just know that without the need to regularly verify the IP and MAC address of the router, access point, or switch your competitive device is connected to, you’ll be able to squeeze a few milliseconds every couple of minutes while maintaining a stronger security posture. And to go even a step further, ensuring that no other ARP traffic along with any other not required traffic is able to be sent to your competitive machine, as your operative system firewall will block it, you may just dodge any network misshapes or less ethical adversaries (in fact criminals, it’s not fun, it’s a crime). How to Speed Up and Protect Your Device? You need to know the MAC/IP address pairing of the router, switch, and access point that your device is connected to; You need to know the MAC/IP address pairing of the router, switch, and access point that your device is connected to; 2- You need to set a static ARP entry for it; 3- You need to enforce robust firewall rules that will allow you to focus all traffic on the task at hand. How to do the above? Well, first, I hope you appreciate and enjoy that I’ve already shared a secret (give the article some love in exchange will you?), so don’t expect me to share everything as perhaps one day we may compete against each other. And unless system tuning is against the rules, I intended to make use of it. Besides, I’m quite sure you can figure it out… if you are a seriously competitive person. If all of the above is a bit above your head and you are worried you may become a victim of a criminal during competition, enquire with the organizer of the games if they will have dynamic ARP inspection devices to protect the players. If they do not, point them to this article.

Competitive Gamer Secret: ARP (Address Resolution Protocol)

With its main idea being to showcase traces across various technology stacks, including asynchronous communication via an MQTT queue, I continue to work on my Opentelemetry demo. This week, I added a couple of components and changed the architecture. Here are some noteworthy learnings; note that some of them might not be entirely connected to OpenTelemetry. Here's an updated diagram. New components appear in violet, and updated components appear in green. I want to be able to add more components. Thus, I decided that instead of directly querying the database, the inventory component would query warehouses, which are supposed to be located in different regions. Each warehouse can be implemented in a different stack, and you can have as many as you want—PRs are welcome. I miss Elixir and .Net at the moment. The contract, which I need to write down, is easy: An endpoint /stocks/${productId}


The ability to query PostgreSQL


Return the stock in the form: I've written about the changes in the inventory to account for the new configuration. Let's talk about the warehouse components. The Go warehouse Let me be blunt: I dislike (hate?) Go for its error-handling approach. However, with close to zero knowledge of the language, I was able to build a basic HTTP API that reads from the database in a couple of hours. I chose Gin Gonic for the web library and Gorm for the ORM. OpenTelemetry provides an integration with a couple of libraries, including Gin and Gorm. On the Dockerfile side, it's also pretty straightforward. I skipped optimizing the mount cache and the final base image, though I might return to it later. All in all, that's the component I developed the fastest. I still dislike the Go language, but I begrudgingly understand that developers who want to get things done use it. The Ruby warehouse While Ruby is not this famous anymore, I still wanted the stack in my architecture. I eschewed Ruby on Rails in favor of the leaner Sinatra framework. I use sequel for database access. The dynamic nature of the language was a bit of a hurdle, which is why it took me more time to develop my service than with Go. I also spent a non-trivial amount of time on auto-instrumentation. For stacks with a runtime, auto-instrumentation allows developers to go on their merry way, oblivious to any OpenTelemetry concern. At runtime, the Ops team adds the necessary configuration for OpenTelemetry. For example, we achieve this with a Java Agent on the JVM. I expected the same "carefree" approach with Ruby, but I couldn't find anything related to the stack. Ruby on Rails has a built-in plugin system, but not Sinatra. I tried to use bash to glue files together, but to no avail. If you're a Ruby expert or have any experience doing this, please let me know how. The GraalVM native warehouse This one is a regular Kotlin application on Spring Boot with a twist: I'm using GraalVM native image to compile ahead-of-time to bytecode. This way, I can use a tiny Docker image as my base, e.g, busybox. It's not as efficient as Go or Rust, but it's a good bet if tied to the JVM. OpenTelemetry did work on the JVM version but didn't when I compiled it to bytecode. Given the compilation time, it took me a couple of days of back-and-forth to make it work. The reason is simple: Spring Boot relies on auto configuration classes to activate or not features. Some auto-configuration classes rely on the presence of classes, others on the presence of beans, others on the opposite, others on existing properties, others on a combination of the above, etc. In my case, the guilty class was OtlpTracingConfigurations.ConnectionDetails. It relies on the management.otlp.tracing.endpoint property: class OtlpTracingConfigurations {

  @Configuration(proxyBeanMethods = false)
  static class ConnectionDetails {

    @Bean
    @ConditionalOnMissingBean
    @ConditionalOnProperty(prefix = "management.otlp.tracing", name = "endpoint")
    OtlpTracingConnectionDetails otlpTracingConnectionDetails(OtlpProperties properties) {
      return new PropertiesOtlpTracingConnectionDetails(properties);
    }
} If the property is not present at compile-time, the Spring Framework doesn't create a bean of type OtlpTracingConnectionDetails. Through a chain of missing beans, the final binary doesn't contain OpenTelemetry-related code. The solution is easy: set the property to an empty string in the application.properties file, and override it to its regular value in the Docker Compose file. While auto-configuration is a compelling feature, you must understand how it works. That's the easy part. However, it's much more work to understand the whole chain of auto-configuration activation regarding a feature. Having distanced myself from the JVM, I'm no longer an expert in these chains, much less the OpenTelemetry one. I finally understand why some developers avoid Spring Boot and name it magic. Migrating from JavaScript to TypeScript I used JavaScript in my first draft of a subscriber to the MQTT queue. Soon afterward, I decided to migrate to TypeScript. JavaScript code is valid TypeScript, so a simple copy-paste worked, with the addition of @ts-ignore. However, when I tried to fix the code to "true" TypeScript, I couldn't see any OpenTelemetry trace. As for GraalVM, I went back and forth several times, but this time, I decided to solve it once and for all. I migrated code line by line until I isolated the issue in the following snippet: const userProperties = {}

if (packet.properties && packet.properties['userProperties']) {
    const props = packet.properties['userProperties']
    console.error('Props', props)
    for (const key of Object.keys(props)) {
        userProperties[key] = props[key]                         //1
    }
} The TypeScript compiler complains with the following error message: TS7053: Element implicitly has an any type because expression of type string can't be used to index type {} I earlier tried to fix it with the following: const userProperties = new Map<string, any>() It compiled, but my limited understanding of JavaScript prevented me from realizing that a Map is not the same structure as an object. I understood the issue only when I isolated the exact line that went wrong. I just had to find the correct syntax to declare the type of an object: const userProperties: Record<string, any> = {} Adding a Redis cache So far, my services have used only PostgreSQL as a data store. Datastores don't implement OpenTelemetry by themselves, but the correct instrumentation of an app can show the trace going to the datastore. Here, you can see the trace created by the OpenTelemetry agent on a Kotlin/Spring Boot app that uses the PostgreSQL driver. Here's the one from the Gorm framework, instrumented manually. Both traces display the system, the statement, and a couple of other data. The Redis instrumentation shows the same information under the same structure! Icing on the cake, if you use the Lettuce client, which is the default for Spring, you don't need additional changes. The OpenTelemetry agent already takes care of everything. Another Apache APISIX instance Last but not least, I've added another APISIX instance. Most organizations manage their APIs from behind a single multi-node API Gateway. However, it can be a significant burden depending on how the organization structures the teams. When a team needs to deploy a new or change the routing of an existing non-front-facing API, they want the change ASAP. If the team in charge of the centralized API Gateway doesn't respond to tickets, it slows down the API team and the business value they want to deploy. For this reason, it's a perfectly valid pattern to set up API Gateways that are not front-facing under the responsibility of each API team - or business department. The granularity here depends on what works for each organization. It reduces friction when the infrastructure needs to change but does so at the cost of more diversified work for the API team. It also allows for different API Gateway technology (or technologies) from the front-facing one. In my demo, I assume the team responsible for the inventory component has set up such an instance. It handles all routing to the warehouses. All other components still rely on the primary APISIX instance. Conclusion In this post, I've described several changes I made in my OpenTelemetry tracing demo and the lessons I learned. I want to add additional warehouse components in other stacks. What stack would you be interested in? Would you like to contribute to such a component? To go further: End-to-end tracing with OpenTelemetry
Improving upon my OpenTelemetry Tracing demo
Parsing structured environment variables in Rust The complete source code for this post can be found on GitHub: https://github.com/nfrankel/opentelemetry-tracing?embedable=true Originally published at A Java Geek on June 2nd, 2024 With its main idea being to showcase traces across various technology stacks, including asynchronous communication via an MQTT queue, I continue to work on my Opentelemetry demo . This week, I added a couple of components and changed the architecture. Here are some noteworthy learnings; note that some of them might not be entirely connected to OpenTelemetry. traces asynchronous communication Opentelemetry demo Here's an updated diagram. New components appear in violet, and updated components appear in green. New components appear in violet, and updated components appear in green. I want to be able to add more components. Thus, I decided that instead of directly querying the database, the inventory component would query warehouses, which are supposed to be located in different regions. Each warehouse can be implemented in a different stack, and you can have as many as you want—PRs are welcome. I miss Elixir and .Net at the moment. The contract, which I need to write down, is easy: inventory An endpoint /stocks/${productId} The ability to query PostgreSQL Return the stock in the form: An endpoint /stocks/${productId} An endpoint /stocks/${productId} /stocks/${productId} The ability to query PostgreSQL The ability to query PostgreSQL Return the stock in the form: Return the stock in the form: I've written about the changes in the inventory to account for the new configuration. Let's talk about the warehouse components. The Go warehouse Let me be blunt: I dislike (hate?) Go for its error-handling approach. However, with close to zero knowledge of the language, I was able to build a basic HTTP API that reads from the database in a couple of hours. I chose Gin Gonic for the web library and Gorm for the ORM. OpenTelemetry provides an integration with a couple of libraries , including Gin and Gorm. On the Dockerfile side, it's also pretty straightforward. I skipped optimizing the mount cache and the final base image, though I might return to it later. Gin Gonic Gorm couple of libraries All in all, that's the component I developed the fastest. I still dislike the Go language, but I begrudgingly understand that developers who want to get things done use it. The Ruby warehouse While Ruby is not this famous anymore, I still wanted the stack in my architecture. I eschewed Ruby on Rails in favor of the leaner Sinatra framework. I use sequel for database access. The dynamic nature of the language was a bit of a hurdle, which is why it took me more time to develop my service than with Go. Sinatra sequel I also spent a non-trivial amount of time on auto-instrumentation. For stacks with a runtime, auto-instrumentation allows developers to go on their merry way, oblivious to any OpenTelemetry concern. At runtime, the Ops team adds the necessary configuration for OpenTelemetry. For example, we achieve this with a Java Agent on the JVM. I expected the same "carefree" approach with Ruby, but I couldn't find anything related to the stack. Ruby on Rails has a built-in plugin system, but not Sinatra. I tried to use bash to glue files together, but to no avail. If you're a Ruby expert or have any experience doing this, please let me know how. bash The GraalVM native warehouse This one is a regular Kotlin application on Spring Boot with a twist: I'm using GraalVM native image to compile ahead-of-time to bytecode . This way, I can use a tiny Docker image as my base, e.g , busybox . It's not as efficient as Go or Rust, but it's a good bet if tied to the JVM. GraalVM native image bytecode e.g busybox OpenTelemetry did work on the JVM version but didn't when I compiled it to bytecode . Given the compilation time, it took me a couple of days of back-and-forth to make it work. The reason is simple: Spring Boot relies on auto configuration classes to activate or not features. Some auto-configuration classes rely on the presence of classes, others on the presence of beans, others on the opposite, others on existing properties, others on a combination of the above, etc. bytecode auto configuration In my case, the guilty class was OtlpTracingConfigurations.ConnectionDetails . It relies on the management.otlp.tracing.endpoint property: OtlpTracingConfigurations.ConnectionDetails management.otlp.tracing.endpoint class OtlpTracingConfigurations {

  @Configuration(proxyBeanMethods = false)
  static class ConnectionDetails {

    @Bean
    @ConditionalOnMissingBean
    @ConditionalOnProperty(prefix = "management.otlp.tracing", name = "endpoint")
    OtlpTracingConnectionDetails otlpTracingConnectionDetails(OtlpProperties properties) {
      return new PropertiesOtlpTracingConnectionDetails(properties);
    }
} class OtlpTracingConfigurations {

  @Configuration(proxyBeanMethods = false)
  static class ConnectionDetails {

    @Bean
    @ConditionalOnMissingBean
    @ConditionalOnProperty(prefix = "management.otlp.tracing", name = "endpoint")
    OtlpTracingConnectionDetails otlpTracingConnectionDetails(OtlpProperties properties) {
      return new PropertiesOtlpTracingConnectionDetails(properties);
    }
} If the property is not present at compile-time , the Spring Framework doesn't create a bean of type OtlpTracingConnectionDetails . Through a chain of missing beans, the final binary doesn't contain OpenTelemetry-related code. The solution is easy: set the property to an empty string in the application.properties file, and override it to its regular value in the Docker Compose file. at compile-time OtlpTracingConnectionDetails application.properties While auto-configuration is a compelling feature, you must understand how it works. That's the easy part. However, it's much more work to understand the whole chain of auto-configuration activation regarding a feature. Having distanced myself from the JVM, I'm no longer an expert in these chains, much less the OpenTelemetry one. I finally understand why some developers avoid Spring Boot and name it magic. Migrating from JavaScript to TypeScript I used JavaScript in my first draft of a subscriber to the MQTT queue. Soon afterward, I decided to migrate to TypeScript. JavaScript code is valid TypeScript, so a simple copy-paste worked, with the addition of @ts-ignore . JavaScript @ts-ignore However, when I tried to fix the code to "true" TypeScript, I couldn't see any OpenTelemetry trace. As for GraalVM, I went back and forth several times, but this time, I decided to solve it once and for all. I migrated code line by line until I isolated the issue in the following snippet: const userProperties = {}

if (packet.properties && packet.properties['userProperties']) {
    const props = packet.properties['userProperties']
    console.error('Props', props)
    for (const key of Object.keys(props)) {
        userProperties[key] = props[key]                         //1
    }
} const userProperties = {}

if (packet.properties && packet.properties['userProperties']) {
    const props = packet.properties['userProperties']
    console.error('Props', props)
    for (const key of Object.keys(props)) {
        userProperties[key] = props[key]                         //1
    }
} The TypeScript compiler complains with the following error message: TS7053: Element implicitly has an any type because expression of type string can't be used to index type {} The TypeScript compiler complains with the following error message: TS7053: Element implicitly has an any type because expression of type string can't be used to index type {} TS7053: Element implicitly has an any type because expression of type string can't be used to index type {} I earlier tried to fix it with the following: const userProperties = new Map<string, any>() const userProperties = new Map<string, any>() It compiled, but my limited understanding of JavaScript prevented me from realizing that a Map is not the same structure as an object. I understood the issue only when I isolated the exact line that went wrong. I just had to find the correct syntax to declare the type of an object: Map const userProperties: Record<string, any> = {} const userProperties: Record<string, any> = {} Adding a Redis cache So far, my services have used only PostgreSQL as a data store. Datastores don't implement OpenTelemetry by themselves, but the correct instrumentation of an app can show the trace going to the datastore. Here, you can see the trace created by the OpenTelemetry agent on a Kotlin/Spring Boot app that uses the PostgreSQL driver. Here's the one from the Gorm framework, instrumented manually. Both traces display the system, the statement, and a couple of other data. The Redis instrumentation shows the same information under the same structure! Icing on the cake, if you use the Lettuce client, which is the default for Spring, you don't need additional changes. The OpenTelemetry agent already takes care of everything. Another Apache APISIX instance Last but not least, I've added another APISIX instance. Most organizations manage their APIs from behind a single multi-node API Gateway. However, it can be a significant burden depending on how the organization structures the teams. When a team needs to deploy a new or change the routing of an existing non-front-facing API, they want the change ASAP. If the team in charge of the centralized API Gateway doesn't respond to tickets, it slows down the API team and the business value they want to deploy. For this reason, it's a perfectly valid pattern to set up API Gateways that are not front-facing under the responsibility of each API team - or business department. The granularity here depends on what works for each organization. It reduces friction when the infrastructure needs to change but does so at the cost of more diversified work for the API team. It also allows for different API Gateway technology (or technologies) from the front-facing one. In my demo, I assume the team responsible for the inventory component has set up such an instance. It handles all routing to the warehouses. All other components still rely on the primary APISIX instance. inventory Conclusion In this post, I've described several changes I made in my OpenTelemetry tracing demo and the lessons I learned. I want to add additional warehouse components in other stacks. What stack would you be interested in? Would you like to contribute to such a component? To go further: To go further: End-to-end tracing with OpenTelemetry Improving upon my OpenTelemetry Tracing demo Parsing structured environment variables in Rust End-to-end tracing with OpenTelemetry End-to-end tracing with OpenTelemetry Improving upon my OpenTelemetry Tracing demo Improving upon my OpenTelemetry Tracing demo Parsing structured environment variables in Rust Parsing structured environment variables in Rust The complete source code for this post can be found on GitHub: https://github.com/nfrankel/opentelemetry-tracing?embedable=true https://github.com/nfrankel/opentelemetry-tracing?embedable=true Originally published at A Java Geek on June 2nd, 2024 Originally published at A Java Geek on June 2nd, 2024 A Java Geek

Optimizing OpenTelemetry Tracing with Multi-Stack Warehouse Components

Stories, News & Mentions about JUSTWORKS

Justworks’s Stories on HackerNoon