AWS Lambda: What Kind of Serverless Applications Can I Make? (Podcast Transcript)

On this episode of the , Amy Tom talks to Pavel Denisjuk about AWS Lambda and serverless applications. Pavel is the Co-Founder and CTO at Webiny. Amy and Pavel discuss Pavel's career in entrepreneurship, building applications with AWS Lambda, the advantages and disadvantages of going serverless, and examples of running AWS Lambda functions with event triggers. Hacker Noon Podcast In this episode, Amy talks to Pavel about: AWS Lambda and its limitations (06:15) Building a serverless application vs an on-prem application (10:07) Planning a serverless application project (16:41) AWS Lambda event triggers and functions (21:20) The technical knowledge required to build an AWS Lambda application with Webiny (25:25) Follow Pavel Denisjuk + Webiny: Follow Pavel on Twitter at @paveldenisjuk Learn more about Webiny at webiny.com Shownotes: Join the Webiny Slack community Podcast Transcript (Machine Generated, Please Excuse The Errors) Amy: [00:00:00] hello, hackers buckle in you're about to have the time of your frickin lives. This is the hacker noon podcast, and my name is Amy Tom. Today I am joined with Pavel. Pavel. Would you please introduce yourself a little bit to the audience? Pavel: [00:00:26] Hi everyone. So I am Pavel Denisjuk, I'm a co-founder and CTO at a company called Webiny. We are building an open-source framework for full-stack application development on top of serverless infrastructure and a micro founders fan. And I have been working closely together for over 10 years running a web agency. We were always looking for. Better ways to build applications, deliver projects faster, with better quality, and then serverless came around and it felt like a natural step for us. So that's how it came to be. Amy: [00:00:59] And can you describe a little bit more about what Webiny does? Pavel: [00:01:03] It's a framework which you use to bootstrap and deploy your applications into AWS. Currently, we are also naming it, supporting other clouds down the line, but at this point we are 100% focused on AWS. So naturally Lambda is our compute and we use other services like dynamo, DB, and S3, of course. CDN like CloudFront, API, gateway and all that, which comes with AWS. And we make it really easy for people who are not very familiar with serverless to get started, because it can be overwhelming when you just Google what serverless is, what a Lander is, then you're like, okay I understand functions, but then you start developing something with, it's like a project or a full blown application where you need rendering and caches and static, snapshots and stuff. And it becomes very complex and it can be really overwhelming for people who are new to this world. So I've been, it takes away that complexity and it gives it a lot out of the box, like page builder, headless CMS, form builder, lots of mechanisms, how you can just. So open the project and start developing new stuff without actually knowing a lot about the whole cloud, the whole serverless thing and learn as you go. Amy: [00:02:22] Cool. How long has it been since you started Webiny? Pavel: [00:02:25] So it's, it is now around two years in development and yeah we pretty much jumped. On the serverless ship, as soon as it kicked off, it was, it just felt natural to us. As we were never a big fans of managing infrastructure ourselves, it takes a lot of effort and knowledge and we wanted to build apps. We want to build business logic and make people's lives easier with software. Right. So serverless, it just took out the complexity of managing infrastructure for us. Amy: [00:02:56] Right. What's your background in prior to Webiny? Pavel: [00:02:59] For the bus 10 years and in Iran, a web agency, so we built dozens and dozens of different client projects. E-commerce maritime, employment, portals stuff like that, like big custom applications web shops. And so we tried. A lot of tools that are currently available on the market from like 2010 all the way to these days. And we were never happy with with the set that is available and that we were always looking, into. Improving our workflows. So that's why serverless was really interesting for me. Right. So you and spin have been partners for over 12 years then? Yeah. Yeah. Yeah. Okay. Wow. They together at college and then we started the business together right after that. Amy: [00:03:47] Right. Yeah. Cool. That's great. It's nice that you have someone that you can trust and someone to build a company with you. Yeah. I think it's almost even like rare to find one person that you work so well with that you can trust and bill is willing to help you build. Yeah, that's great. Cool. And what was your degree in? Pavel: [00:04:13] So I have a master's degree in business information systems. And computer science. So it was like a bachelor's degree in computer science, then business information systems. Amy: [00:04:23] So right out of college then did you both go right into entrepreneurship or Pavel: [00:04:30] Well Sven was the guy for entrepreneurship. He was like all about business, always, although he's really good at tech. I was the one more on the only tech side and spend was doing business. Oh, yeah, he's a coder himself. And he's very much into tech and everything, but, for him, it was just more natural to do business and I was more on the coding side. Cool. Amy: [00:04:56] And, but did you start right after college, on Pavel: [00:05:03] have experienced before starting that we did work for other companies in the area? To gain some, junior level experience. And and that, and after that we immediately started in the trenches building projects, shipping them to clients. And yeah. Nice. That's like the dream, at least for me, but yeah, it was challenging. It still is, but it's nice. It has pros and cons like the job or career path. That's super cool. What was your very first job ever? I believe it was in like in high school. I was renting a like some protective umbrellas on the beach because I'm from Croatia. During summer I was literally the beach boy and blending umbrella. It's just cool to see like how far you've come really. Oh, yeah, I was I was switching jobs, not to, not because I was leaving them, but I was doing seasonal jobs, because of course education was the primary thing. And then I was just I was taking any offer I could get, but it is great. It all builds up to, your character, your work like how you approach work, how serious you are about work. I'm glad I did travel that path. Amy: [00:06:15] Yeah. Right. Okay, cool. So I want to talk to you today a little bit about AWS Lambda then, because that's primarily what webinar is working with. Can you tell me what it does? Pavel: [00:06:29] So AWS Lambda is it's like a virtual machine, which you don't set up and you just upload your code into it and you run it. It takes care of running the entire operating system in the background. You don't need to update any any software, you just deploy your code and it aggregates it almost like magic. And you pick how much memory you need to execute it, what the run time limits are and timeouts and stuff like that. But you never need to set up a virtual machine install a piece of software or anything like that. Amy: [00:07:02] Cool. And what kind of applications might you make with AWS Lambda? Pavel: [00:07:06] So if you think about Lambda is your easy to container or any virtual machine for that matter. It's a simple compute, so you can run anything on it. There are of course limitations. So I think it's easier to explain. What's not so suitable for Lambda than what is so let's go from that direction. So what's not very suitable for Lambda are long running. Operations because Lambda has a maximum run time limit of 15 minutes. It means that if you have like video transcoding or something going on, which is taking a long time, you can't do that in a single land execution because it will simply shut down. Long running operations are not the best for Lambda. And then you need to look into containers or building self invoking Lambdas, which is just not, it's just not convenient. So that's one type of applications and the other is web sockets. Like it can work, but it becomes very expensive at scale. Because to implement WebSockets you also need to coordinate it with API gateway, dynamo DB. So it's not just Lambda. You need to start combining different resources. And then it all adds to the total cost. And in the end for really big applications that have massive traffic, it becomes more expensive than running a virtual machine. Amy: [00:08:28] Right. Okay. So are there any restrictions of coding, serverless applications, anything else? Pavel: [00:08:35] Recordings of serverless apps requires the fundamental mind shift. It's not the same how you approach serverless applications versus how we use decode when you have PHP or Python, node, but in, in your regular virtual machine environment for serverless. You need to think more in terms of events and how we are going to connect your function with other services, all our databases, especially non-service databases currently on the market, there are not so many serverless databases. They're like three or four. There are slowly catching up and beginning to develop, but. Like for AWS, it's only natural tools, diner, would you be because it is actually serverless in it. It doesn't require a connection to be maintained between Lambda, which is currently running and the database saying endpoint. We did have some experience running Mongo DB. And we have a huge blog post on revenue blog about it and how we try to overcome some of those limitations and problems. But it it, it does require a lot of hacking and engineering around the problem instead of the problem being solved by the provider. So would it be, is a solution to that problem because it's available right there in your AWS cloud. You don't need connections. And you pay for what you use, so you don't need a database cluster running in the background. You just send a request for data or you write the data to it. And it scales infinitely just like Lambda does. So it's a perfect match. Amy: [00:10:07] Okay. So if I'm looking to build my application, why would I want to go with serverless versus on-prem? Pavel: [00:10:15] Yeah. There are of course pros and cons to both. The main advantages of serverless is of course that you have no hardware to maintain. You just don't need to worry about the hardware is sorted for you by the cloud provider. There is no software to update. You don't need to manage patches and updates to your database service or your operating system. No matter how hard Users hate your service, be it an API or a website, and the cloud provider will scale automatically for you and then downscale. If there is no traffic, you don't need to worry about it. If your blog post ends up on death crunch and a million users hit your website, you don't need to worry about. Will my website survive this onslaught of visitors because it will, AWS will take care of it, take care of it. It will upscale and downscale for you, and you will just pay for the amount of requests served by your Orlando. So in a way, then you're also saving money because you're not paying. So like a bucket of services you paid on what you want. I worry about having a 10 virtual machines behind the load balancer, just in case, your block balls goes viral or something. You don't need to do that. And then. It may never go viral, but you're still paying for it with serverless. You're solving that problem and you're not overpaying and it will never fail because it will automatically scale. So that's on the advantages side, right. But there are industries that have very strict regulations around data privacy and security in the cloud. In general, right? Because serverless is powered by the cloud provider. So it means that it runs inside of some public cloud. So governments or banking, they will not be so eager running on public cloud. And there are also many companies that operate under some form of regulatory control because they handle. Private health information, student records. There are other industry regulations that just limit and define how you control your data. And those companies must always know whether data is at all times while in the cloud. You can never tell you have an endpoint or an API, you can ask for data, but can you really say where your data is? And who has access to it. In that case, those companies nowadays often opting for a hybrid cloud approach were the most sensitive parts of the system that are strictly regulated around on-premise and they have their own data centers and they handle it themselves. But those less strict bark are. Hosted in the cloud and then they can, of course you verbalize or traditional virtual machines or whatnot. But the point is there is this just difference in security and data, privacy control and stuff like that, which in the end, is that the decisive factor for going to cloud or staying on prem. Amy: [00:13:15] Okay. Is it possible to use AWS Lambda on private cloud? Pavel: [00:13:21] Interesting story which will answer the question. We are talking to numerous enterprise clients and on the topic of data regulations, right? They have a strict requirement that their data stays within their data center, but every requirement is we will, we have a complete. Replica of AWS environment, local in our data center using Kubernetes. So they wanted to use Lebanon because it only runs on AWS, but to run it on their local data center, which means that if you just think about a Lambda beings, something that executes your code, then you can effectively run it anywhere. It w it wouldn't effectively be AWS Landa, but it would still be like a function as a service. Yeah. Oh, yeah, people are building all kinds of crazy things nowadays. And even this private clouds thing, what were the simulated services, which are 100% compatible with AWS, it's crazy. And it's amazing because the amount of things that unlocks this. That's really great. Amy: [00:14:27] Okay, cool. I have another question to follow up that as well then. Is it possible to be, say like GDPR compliant on public cloud then? Pavel: [00:14:39] Yeah. Funny thing is that I I personally don't know because webinar isn't a SAS or anything like that, we only develop the framework and people are hosting their applications inside their own accounts on AWS. So we never had to worry about. About th that type of restrictions or control of data privacy. So we let our clients or users, our users and all that for themselves, the ones that are sensitive to that. Amy: [00:15:10] Yeah, I think it must be in some sort of way. There must be some kind of like security control in public cloud that helps them or yeah. You can go for the hybrid cloud option as well. That's really interesting. So you want to move over to severless? Well, Webiny is an source serverless framework that abstracts away all of the serverless challenges and allows you to easily build solutions on top of the serverless infrastructure it's completely free to use and self hosted. So give it a try and see for yourself at Webiny dot com. Hm. Okay, cool. Can I still make multiple obligations work together when it's not hosted on prem? Pavel: [00:16:00] Oh, absolutely. In the cloud or on prem in the end, it's all some kind of an API or a web server or something that communicates with network. So if you can send a request, then you can communicate with other applications. So yeah, it's definitely not limited. There are no, no kind of like limitations, were problems in that regard. Okay, cool. And what languages are supported with AWS Lambda? So the most popular one is no JS, but Python is also there go Ruby, java.net. Those are the primary runtimes that AWS offers. Amy: [00:16:41] Okay, cool. And how crucial would you say that planning would be for serverless applications? Pavel: [00:16:47] I think blending is quite important for any kind of application development for serverless it. It really depends on what you're building, especially in the early stage. Like you can get away with not planning too much at like minimum viable product stage. If you're just developing a proof of concept, which you need to show to you, or I dunno investor or a potential client in a month, you could start pretty immediately without. Very rigorous planning. Especially if using tools like webinar, which sets everything up for you, then you can start writing your business logic immediately. But planning does become important at a stage where, your product is going to grow. And then at that point, you need to plan for amount of requests you plan to be getting is your application more of a read nature or write nature, meaning do just serve data or will there be a lot of rights to your API? Those operations heavily influenced how you architect things in the background, especially if you work with elastic search which is not serverless. And you need to handle that kind of traffic in an asynchronous way and add the things like queuing into the mix. So in that regard it can become complex. So yeah, planning is definitely important that, when you're ready to build a production application, Amy: [00:18:10] Right. And then what about with auto scaling is let's say I knew that my tech crunch article was coming out that would drive a lot of traffic to my website. What I need to plan anything for that or does it just happen all automatically? Pavel: [00:18:26] It just happens. It just happens. So say you get loads of traffic on your website or. Most of those requests will be served from CDN, from CloudFront, but the ones that need to hit your API and in the end, that request lends on your Orlando. Once. AWS sees that there are too many requests. It won't just shut down. It will instead spawn more Lambdas to handle all those requests. Two, three, 400, 200. It doesn't matter. It will just keep spawning unless it can satisfy the demand. And once all the requests are settled and served AWS monitors it for you. And if no more requests are coming in, it will just shut down the remaining Lambdas. If they're not required by it, so it's just happening. It's like magic. Amy: [00:19:17] Right. Okay, cool. Can you give an example of what someone might make with webinar? Pavel: [00:19:25] Yeah. Out of the box with the webinar, we provide a page builder think WordPress, in a sense, right? It's a visual page builder, which you can use to build landing pages and even write blog posts and stuff like that. Then we have a headless CMS, like stroppy or Contentful. You can model data right there and we expose it for a graph QL API. So there are many tools available out of the box for you to use and combine. And besides that we provide a full set up for developers, so they can immediately start working on custom stuff and build their own APIs. Maybe they are developing a mobile application, they need a custom API. They could use a headless CMS for that, or they can go in and build. Custom data models and use dynamo to, to store that data to AWS. And it will all be exposed via graph QL. It's a framework to build anything you need, but we provide really good basics and several apps out of the box. And one of the, one of the really useful apps is the file manager as we call it. It helps you handle upload and handled and transform resize images and It's integrated with all other apps, like headless CMS and page builder. So you get a good package to start with out of the box. And we also handle things like pre rendering. So yeah, your page is so each of your pages will be rendered into a static, HTML, snapshot, and stored in a cache. You are pretty much serving static content all the time. Almost like Godspeed, if you want something, to, to compare to except God's been needs to rebuild the entire application on every change. And we took the pre rendering approach where you're only rendering things that are changed. And we saw that and there, of course, an API expose for that. So developers can utilize that service of followers too. Built more on top and use it for other use cases. We may not even predict. Amy: [00:21:20] Okay. I think I'm trying to like wrap my head around what an event could be, what the function might be, what it might connect to you. So what's an example of that. Pavel: [00:21:31] In terms of AWS, almost everything is an event, but if you translate it to an older way of doing things, that would be your HTTP request. And an HTTP request would travel to a CDN then to API gateway. And then API gateway knows that particular request needs to land on a Lambda. And then it acts it into a payload which they call event. And your function is then executed with all the data and metadata that the request and you get it in your Lambda function to process, and then you can do whatever you want with it. But in case of a API call, for example, you would load data from database and then return it to to the requester. Right? An event is just a payload to a function which will be processed within the function itself. Okay, but this is very technical as well, right? Like your explanation is very technical, which is great. But I also want to know like a real life example of how this might work. Okay. Yeah. Let's say you are requesting a a file. Let's say it's a lot like a logo for a website. That request will travel to a CDN. And then for example, you want your logo to be. Of exact size, let's say 300 pixels, but in your system it's uploaded as a larger image. So we need to resize it, but we need to downsize it. In that case, your your request will land on a Lander function. And in the event, like as a payload to do that function, you will receive information about the. Actual request to the file path. Like you would receive it in an express application for no JS or in any PHP framework, it's it's simply a date. Yeah. So bald the issue to be request, and then you parse that request yourself. It's your problem, how you parse it and you extract the important pieces of the request and instruct your code to transform it somehow. Again, it's all up to the developer, what he wants to do with it. And then you return the new downsized image as a response to that request. And it's the same beat and HTML file. Maybe it's just some Jason data. The request off is pretty much the same. It always goes CDN, API, gateway and Lambda and inland others. Your code is being executed, which does your magic. Amy: [00:23:53] Right. Okay. So the event is that someone uploads the logo as the image, and it recognizes that the function needs to downsize the image. Pavel: [00:24:03] Yeah. You can put it that way. Like any event is a simple HTTP request, right? Which in terms of AWS, it becomes any event. That's all they call it. And for Lambda function, everything is an event. And because when that can be triggered, In different ways. It doesn't have to be only HTTP requests. It can be a file being uploaded directly to S3 bucket. It can be a dynamo DB stream, which is handling lots of records. And then it. On every record, it can involve a Lambda function to process that record. It can be a, an SQS trigger. There are many events and events sources in AWS that can trigger land even even Kognito. For example, if you want to verify Or modify a JWT token on Cognito, for example, where somebody logs in there is a bunch of triggers you can configure. And then during the authentication flow, those triggers will be executed. And then you can do some logic augment the data on the JVT token or deny a user access, stuff like that. In terms of AWS, Lambda is like a glue. Whenever you need some custom logic down somewhere or connect services. You can link slender with many services as event sources and it will be executed with a specific payload from that service. Amy: [00:25:25] Okay. Got it. Got it. So you mentioned that with webinar, you don't need a lot of it. It's easy to use when you're a beginner, right? What's the baseline technical knowledge that you would need to start building your serverless application with webinar. Pavel: [00:25:43] Okay. So the basic tools are no JS. Definitely. Everything is written no JS script, which makes it also slightly easier. Graph QL is our go-to tool. So our APIs are all exposed by a graph QL. So knowledge with that will definitely help. And on the client side we use react. So those three tools can get you started. Okay. And if I wanted to learn more advanced functions with webinar, what might I look at learning as well? We are providing guides and tutorials on our docs where we really have step-by-step instructions and stuff. And then we also provide links to technology used in the background. So our docs are definitely a good place to to start, but regarding technology itself, like the tools themselves that we use we use as our infrastructure as code tool which is. Which is great. It allows us to deploy infrastructure to different clouds, not just AWS, which is important for us in the long run. And it defines infrastructure using code, which is a lot easier to read for developers than Yamhill config files. It's a lot clearer. You just follow the code and you can exactly see which resources will be deployed to AWS. I wouldn't say you need a lot more knowledge in regards to technologies themselves, but just getting to know the system because the system is based on plugins and there are many plugins. So it's just the process of discovery, in grasping the whole thing, how stuff is connected, that thing can be overwhelming at the beginning, but we have a great community. We have almost 800 community members on Slack. And the team is present there all the time. So if anybody wants to give up any ago feel free to join our Slack. We are very responsive and the community is great. So there is no question is left unanswered, and we will always join and help with pointers and references and you know how to resolve problems, if any. Amy: [00:27:52] Great. What kind of advice would you give to first-time serverless application developers? Pavel: [00:27:59] That's a good one. That's an interesting one. Yeah. I wish I had someone give me advice on that when we started. I would just say, take it step by step cloud in general. Is complex and cloud vendors have their own terms for things that are effectively the same across cloud providers. And it can just be intimidating. If you just start reading blog posts, you will run into dozens of terms that are completely unfamiliar to don't feel discouraged. Definitely not take it step by step and yeah, just start building something because you'll learn best when you build it. And it was the same with webinar. Two years ago landfills had so many problems with cold starts and no no blog posts about how to use it. We had to experiment a lot ourselves nowadays like I'm saying nowadays, like it's 10 years ago, it's just two years. It developed so much. It's crazy. There are many resources. There is a cloud group educational platform where you can learn a lot about. All clouds and you just need to start getting your hands dirty. That's all. Just start learning, start building small stuff, then expand. And yeah, you'll get there. There is unfortunately, no really easy way to just magically learn everything you just need to build. Amy: [00:29:24] Great. I, so I know that AWS Lambda was first announced or launched back in 2014. Why do you think that it got so popular so fast in the past few years? It solves a problem of quick experiments and prototyping especially for freelancers and small companies who just need to build something quickly. Pavel: [00:29:47] Lambdas was an amazing sandbox, which I love to deploy our code executed and forget about it because you're not paying for your code sitting there, right. Lambda, you upload your code and if you don't use it, you never pay. So it's amazing. Like for developers, you can flush it 10 ideas in one hour just by using Landon and not spinning up server and then sinking stuff with our sink or FTP or whatnot. So it's. I guess the speed of iteration, like how fast it iterate over your ideas and your code is what made it popular in the end. But of course all the advantages we mentioned, like auto scaling, pricing the way How you, how it's priced and just that fundamental maintenance and effort around having a time it's all gone. I can just deploy my code. So I think that's that's one of the key things that really made it popular. Amy: [00:30:43] Right. Okay. Pavel, thank you so much for joining the podcast. If our listeners want to find you, where can they look? Pavel: [00:30:52] You can find me on Twitter. Spelled exactly as my name and on the web.com communities like I'm hanging out there seven days a week. So come chat with me and ask questions. I'll be glad to answer and help. Amy: [00:31:06] Okay, perfect. Thank you very much. Pavel: [00:31:09] Thank you. Amy: [00:31:10] If you like this podcast, don't forget to subscribe. Ray to five stars on tell your friends, you can find us at hacker noon on Instagram, LinkedIn, and Twitter. And I will talk to everyone next time. Thank you very much.