CloudNuro.AI is a next-generation SaaS company focused on bringing artificial intelligence and machine learning to SaaS operations and management, optimizing SaaS spend, improving security and compliance, developing SaaS insight, and maximizing ROI for SaaS investment. CloudNuro offers a central point from where, apart from discovering the growing stack of SaaS applications, software administrators can manage application access for users/teams, licensing and spending, IT workflows, integration, data and access security, SaaS related process automation and policy compliance. CloudNuro is used by companies to realize the following benefits: 1. Discover their growing SaaS application stack for greater control and visibility over their SaaS portfolios 2. Track utilization, manage costs and eliminate wastage in the form of unused user license for SaaS applications. 3. Automate provisioning and de-provisioning of users for added security and process compliance 4. Centrally manage SaaS licenses and renewals 5. Uncover "Shadow IT" Applications (Use of SaaS systems/services without explicit IT department approval) 6. Ensure transparency and accountability of SaaS application usage across all employees We've partnered with Google (GCP) to provide enterprise-grade cloud solutions and Cloud Security Alliance (CSA) to help ensure to bring the best practices for a secure cloud computing environment for all our customers.

CloudNuro

cloudnuro

is one of the most popular CMSs. It’s easy to install and easy to use, and there are thousands of plugins and templates that make the website development super easy and super fast. But its advantages are also disadvantages because the page generation can be really resource-demanding and WordPress is not only one of the most popular but also one of the most vulnerable systems. WordPress There are many cases where a simple static webpage would be enough, but using WordPress is a more convenient solution.  A blog or a product website is a good example. You have to simply choose a template, set the colors and some other things, upload the content, and voila, you have a full-featured website with some hours of work. These are the cases when CloudFront can be a good solution to make your website fast, scalable, and secure. is a . A worldwide distributed cache system. When a user tries to access your content through CloudFront, the system routes the request to the nearest edge server which serves the content if it exists on it or asks it from your WordPress server. Amazon CloudFront Content Delivery Network In this case, WordPress is something like a static page generator that generates the pages that CloudFront serves. The resource consumption of your site will be extremely low because CloudFront asks for the content only once and serves it million times if needed. You can handle extremely high traffic with millions of users by using only a micro instance for your WordPress! Another advantage of this system is that only the CloudFront part is public. Your WordPress is hidden from the outer world so, nobody can attack or hack it. Behind CloudFront, your origin is in maximum safety. To add CloudFront to your WordPress site, set up a CloudFront distribution on the AWS console. Set the origin domain to your current WordPress site URL, and give a name to your distribution. On the “Settings” block choose an alternate domain. This will be the new domain where your WordPress site will be accessible through CloudFront. If the deployment is done, copy the CloudFront domain name, and create a CNAME record to it. For example, if your alternate domain is and the CloudFront domain is , then add a www.example.com d3qu6ceheilwux.cloudfront.net www.example.com CNAME d3qu6ceheilwux.cloudfront.net record to your DNS zone file. If everything has done well, you will see your site on . Clicking on the links on the new site will go back to the original WordPress site. To prevent this, add these two lines of code to the beginning of your wp-config.php: www.example.com if($_SERVER['HTTP_USER_AGENT'] == 'Amazon CloudFront') {
    define( 'WP_HOME', '... new site URL …' );
    define( 'WP_SITEURL', '... new site URL …' );
} else {
    define( 'WP_HOME', '... original site URL …' );
    define( 'WP_SITEURL', '... original site URL …' );
} The user agent of CloudFront is ‘Amazon CloudFront’, so when CloudFront reads the site content, it will get the new URLs, but when we access it for the administration, it will use the original URLs. After this modification, the links will point to the CloudFront URLs (ex.: ). www.example.com Congratulation! You have set up CloudFront for your WordPress site. We are almost done, but I promised improved safety and our original site is still public. The easiest way to prevent attacks on the original site is to add HTTP authentication with a strong password to it. If you are using Nginx, you can generate a password file by using the htpasswd utility: htpasswd -c /etc/nginx/htpasswd wordpress that you can add to your config: auth_basic           "Password protected content";
auth_basic_user_file /etc/nginx/htpasswd; CloudFront can send HTTP headers when it accesses the origin, so you can add the auth header to its config. The auth header looks like this: Authorization: Basic {base64encode({user}:{password})} So, if your user name is abc and your password is 123 then the auth header will be: Authorization: Basic YWJjOjEyMw== The header can be generated by using any . If you generated the header, add it to your origin’s config. online base64 encoder After the deployment, CloudFront will access your site, so it will be publically available through CloudFront, but the original site will be password protected. To keep your site protected, you can change the password monthly and set up to prevent brute force attacks. fail2ban The last thing that I wanted to talk about is invalidation. The default behavior of CloudFront is that it refreshes the content only once per day, so if you change something and don’t want to wait for a day, you have to delete the cache. CloudFront uses invalidations to do it. If you want to delete one or more files from the cache, you have to create an invalidation. Invalidation is a broadcast message to the edge servers to delete the given content. The parameter of the invalidation is a list of file paths that can contain * If you want to fully delete the cache, use ‘/*’. . As you see, CloudFront is a perfect solution to scale your WordPress site and make it safe. By using it, your site can serve millions of users, and you need only a micro instance behind CloudFront. The solution is not WordPress specific. You can use it for any site that does not use server-side interactions. In some cases, it needs a little bit of different thinking. For example, you have to use hosted comment services like Disqus or Facebook comments, or if you have a webshop on your site, you have to organize it to another domain as a separate service. But I think it's a really low price for the safety and the seamless, fast, and scalable content serving.

How to Make Your WordPress Site Safe and Fast With Amazon Cloudfront

How Amazon CloudFront Secures Content Delivery

CloudFront is AWS own CDN (Content Delivery Network). CDNs are primarily used for caching, and many customers also use AWS CloudFront CDN as a security layer, or use it to handle network spikes. With AWS CloudFront CDN, when a user requests a webpage or an image, the request is routed to one of Amazon’s locations. If the edge server already has the resource cached, it’s served to the client. If the resource isn’t on the edge server, it makes a request to a larger edge server called a regional edge cache, if it’s not there, the request arrives at an origin server that houses your application. The edge server then saves a copy of the response locally so it can handle the next request without pestering the origin server. This reduces load on the origin server, helping you keep the instance housing your application small, and is able to reduce latency for clients by moving commonly requested resources closer to the requestor. This is great for your latency, and user experience, but how does it impact your costs? 225+ edge server This guide will walk you through the following: AWS CloudFront pricing Factors that determine Amazon CloudFront cost Ways to Reduce CloudFront Cost AWS CloudFront Pricing AWS CloudFront pricing starts with a very simple pricing model, you’re charged per GB transferred to the internet and for every HTTP request. There are also lots of optional features, like encryption, logging, Origin Shield, file invalidation, and edge server Lambda invocations that can also . incur additional CloudFront costs CloudFront Cost & AWS Data Transfer Cost The AWS CloudFront cost is billed per GB transferred from any CloudFront edge server out to the internet. Once you start transferring more than 40TB of data, you are charged less per GB, and the cost continues to decrease the more data you transfer until you hit 5PB. Like most AWS services, these costs vary by region. The initial rate for the serving data to the cheapest regions (United States, Mexico, Canada, Europe and Israel) is $0.085 per GB, and this cost doubles to $0.170 when serving data to the most expensive region (India). You’re also charged for requests that send data to your origin application, like WebSocket connections, or data attached to PUT requests from a client. Because CloudFront is intrinsically global, the region you’re billed for isn’t determined by where your application lives, but by where the requests to your application originate. You can manage these costs by setting a on your CloudFront distribution that excludes certain regions from your edge server network. price class You can get an Amazon CloudFront pricing discount by committing to a regular level of use (typically at least 10TB transferred per month for a year long commitment period). Unlike many other services, these CloudFront “Reserved Instance” prices are not published, but if your organization can reasonably commit to transferring that data over a long time period, it could reduce your costs, so and ask. You can also get a 30% discount by signing up for the . You have to commit to spending a specific amount on CloudFront services every month for a least a year, and it’s a great deal if you’re also considering utilizing the Amazon (WAF) because 10% of your commitment value is generated as WAF credit. contact AWS CloudFront Security Savings Bundle Web Application Firewall CloudFront Request Costs Another factor of AWS CloudFront pricing is being charged per 10,000 HTTP requests, and you’ll pay a premium of about 25% to handle HTTPS requests. These CloudFront costs also vary based on the region the request originated in, but the pricing difference is less dramatic than for data transfer, dropping from a 100% premium for data transfer to the most expensive regions to a 10% premium for handling requests from these areas. Other CloudFront Costs is an additional caching layer that operates (and is priced based on) the availability zone of your origin server. All requests to your origin server go through this single caching layer, so users accessing different edge servers in different locations can be served the same cached content, and applications operating multiple CDNs have access to a unifying caching layer. This service is charged at the same rate as HTTP requests, adding to the cost of any request that makes it past an edge server. CloudFront also allows you to so only applications that have access to the decryption key are able to see sensitive data. This service costs $0.02 for every 10,000 requests that are encrypted (in addition to the HTTPS request costs). You have the option of generating and monitoring at a cost of $0.01 for every 1,000,000 log lines written. Finally, you can provide custom for your CloudFront distributions for $600 per month. Origin Shield encrypt specific fields real time logs SSL certificates and domain name How to reduce AWS CloudFront Costs Now, with the understanding of Amazon CloudFront pricing and the many cost factors that contribute to its usage, there are several practical and common methods to reduce CloudFront costs. In the following, we’ll cover ways to reduce S3 and EC2 costs with CloudFront, the best way to configure CloudFront, and how to increase your cache hit ratio to trim costs. Reducing AWS S3 and EC2 Costs With CloudFront One of the most common uses of CloudFront is delivering web and content stored in an S3 bucket or EC2 instance to clients all over the world. At low data volumes, the cost of using S3 is actually cheaper because every month you get the first GB transferred out free, but as soon as you start to increase your usage, the cheaper per GB cost of serving data from CloudFront kicks in. In a cheap region like US East Ohio, that inflection point is 18 GB. media Once the amount of data transferred out to the internet increases, the pricing gap for data transfer costs starts to widen. Once you start transferring terabytes of data, the distance between serving content from an origin like S3 and using CloudFront to deliver content opens up dramatically. It’s important to realize that CloudFront has a special status in the AWS universe. Normally, you’re charged for data transferred between availability zones. However, data transferred from AWS services to CloudFront servers doesn’t cost a cent. If you’re transferring a very small amount of data out to the internet, it’s cheaper to not use CloudFront, but if you’re transferring data between zones, or are dealing with larger data transfer costs, CloudFront is almost always going to save money. This special status also makes it cost effective to always pair CloudFront with hosting your origin server on the AWS network. If you self host your origin server, or use a different provider, those data transfer savings vanish, and you have to pay to transfer data into CloudFront, which means anytime an item is not in the cache, you’re paying twice, once to transfer data to the CDN, then again to transfer the data to the client. Comparing the cost of handling HTTP requests between S3 and CloudFront is less straightforward. If your application is exclusively serving GET results, handling requests out of an S3 bucket is significantly cheaper. In US East Ohio, you’ll pay $0.004 per 10,000 requests made directly against an S3 bucket, but you’ll pay at least $0.0075 to serve requests from CloudFront, making it almost 90% more expensive. This calculation is flipped once you start mixing in POST requests. The price doesn’t change based on request type for CloudFront, but it shoots up by more than 100% for S3 buckets, you’ll pay $0.05 to serve up 10,000 POST requests, so CloudFront becomes the better deal, offering an 85% discount on these types of requests. For most applications, data transfer costs and savings on instance size will significantly overshadow the cost of serving requests, so this difference is rarely significant, but if a large portion of the costs for your CloudFront application are caused by many requests for small files, and you can guarantee those requests will primarily be GETs, then consider skipping CloudFront. On the subject of requests, you should also evaluate how necessary using HTTPS is for your application. Handling HTTPS requests adds about 25% to request costs, so if you’re not receiving sensitive data, using plain HTTP can save you a chunk of change. Configuring Amazon CloudFront to Reduce Costs One of the best ways to reduce requests to any web server is by caching content on the client’s browser. This basic idea applies twice when using CloudFront, because CloudFront adds a second layer of caching, files are also cached on edge servers. By default, all files served by CloudFront’s edge server expire after 24 hours, which is far too low for many types of content. You can increase the value of your CloudFront network by appropriately caching objects, particularly ones that you are confident will not change for a while. You can set how long both the client and edge server caches objects by setting the Cache-Control HTTP header at your origin server. You can also instruct your edge servers to implement their own caching policy by setting when you create or update your CloudFront distribution. Be aware that edge server caching is important to plan and implement carefully because if you want to invalidate files before they expire, you can invalidate 1,000 paths for free each month but you’ll pay $0.005 after that limit. Time To Live (TTL) values per path invalidated Increasing Your AWS CloudFront’s Cache Hit Ratio A primary goal of the CloudFront CDN is to avoid inundating your origin server by allowing edge servers to efficiently respond to client requests. This functionality is undermined if most clients end up requesting content that isn’t cached. The ratio of requests that are handled by the edge server compared to the number of requests that arrive at the origin server is called the cache hit ratio. You can increase this ratio by understanding how the works. When a resource is requested by a client, CloudFront generates a cache key (by default, this uses the domain of the distribution and the path of the requested object, but you can then store the cache key and use it to serve future matching requests. To increase the hit percentage, consolidate application views to resolve to a single cache key and avoid adding data that might cause two requests with the same results to generate different cache keys, like creating two URLs that point at the same resource. cache key modify those settings Compression to Reduce Amazon CloudFront Cost CloudFront can be configured to some kinds of files. CloudFront compression will result in a faster load time for clients, and will decrease your costs as edge servers transfer smaller amounts of data to clients. In addition to using default compression, compress resources as they are generated by your origin server. automatically compress AWS CloudFront Free Tier The CloudFront free tier is one of the most generous in the AWS family, you get 50 GB of data transfer out and two million requests every month for a year. If you aren’t already using CloudFront, the free tier can help you decide if it can help you downsize the instance serving your application or dramatically improve client experience by decreasing their load time. Conclusion Amazon CloudFront CDN makes serving resources from a globally distributed CDN simple. This can reduce latency, handle spikes in demand, and decrease the load on your application servers. Leveraging good caching policies, compression, and understanding how the cache key works allows CloudFront to perform these tasks effectively, which helps you get the most value from the service. First published here

A Guide to Paying Less For AWS CloudFront

The demand for video is growing, and even more, businesses find infinite possibilities in this sector. We’re not just referring to entertainment or instructional materials here. Content providers, small businesses, and corporate brands are all benefiting from video on demand. Brands can build stronger relationships with their customers by offering them access to the information they want, whenever and however they want it. Thus, such visualization became the most convenient way to share data with software users. VOD (video on demand) refers to any content delivery method that allows users to select when, where, and how they interact with media integration. This can be accomplished by either simultaneous broadcasting from an internet source or by the user downloading the video to a personal device for subsequent viewing. This is in contrast to traditional streaming, when the viewer may only see their film on a gadget with a satellite or cable connection at a certain period. VOD Streaming has a wide range of applications in the business sector. It may be used to make money, connect with people, and more. Although creating video content necessitates a substantial amount of time and work, many experts believe that the investment is worthwhile. VOD streaming data and trends show how valuable this API integration is, for example, the statistics demonstrate, that the websites featuring video content keep visitors on the site for a longer period of time. This factor influences the engagement of the user with the content, and in particular with the VOD streaming – increasing the chances to convert a lead into a client. Why VOD Delivery on AWS? By using AWS Cloud, Amazon Web Services (AWS) offers two media integration solutions for delivering VOD content to worldwide viewers. The Video on Demand on Amazon Web Services solutions provides the resources needed for API integration of sustainable, decentralized VOD transportation and management processes. This solution makes , which decomposes the workflow into individual phases, making it easy to adapt or expand the architecture to meet your unique video-on-demand requirements. use of Amazon Web Services This method reaches multiple functional advantages: Transcode videos uploaded to S3 into formats that can be played on a variety of devices automatically. Upload individual files for each input to customize MediaConvert job settings. Store transcoded files in a destination bucket and send them to end-users through Amazon CloudFront. The AWS solution for Video on Demand may be readily modified and used as a starting point for more sophisticated workflows. The VOD on AWS solution, on the other hand, makes use of AWS Step Functions, which break down the workflow into individual phases, making it easy to adapt or expand the architecture to meet your unique video-on-demand requirements. Getting Started This guide will show you how to set up and configure different API integration services to upload, transcode, and deliver video on demand to your users. First of all, we need to plan the content storage and processing pipeline. It would be a great solution to implement video uploading on serverless architecture. The choice was made by the peculiarity of video content transferring. It could easily cause traffic overhead on any server, and balancing systems in most cases are too slow to react to them quickly enough. It is the reason we use to accept media integration, store it on , and trigger processing with service. AWS Lambda an S3 bucket AWS Elemental MediaConvert Here is the diagram of the system we going to build: What you’ll need Amazon Web Services account Python 3.8 Git 2.0 or newer]( ) https://cdn.hackernoon.com/images/xWx1eln9Ida9r216TnPyHFrKw1J2-fu037xo.png 1. Create User and Storage for Videos Create AWS User First, we need to create an AWS user and buckets for files. To create the user you should follow the following instructions: Sign in to the Amazon Web Services Management Console and open the IAM console. Select Users from the navigation pane, then Add Users. Select the Programmatic access as a type of access. For permissions choose S3, MediaConvert, Lambda, Cloudfront, IAM, CloudWatch Events, and CloudFormation. For tags, you could add whatever you want to categorize the user by. Copy and . We’ll need them later for Lambda to access resources. Access key ID Secret access key Create S3 buckets The workflow we planned to implement needs 2 buckets: one to store videos downloaded by customers, and the other one to store transcoded videos and distribute them to users. So you have to follow the instruction below twice. Go to Amazon Web Services S3 dashboard. Press Create bucket button and type the bucket’s name. Then choose the default region for your account and don’t forget to check the “Block all public access” checkbox, otherwise, there would be security issues in the system. Leave other options as is and press Create bucket button Buckets should be created in the same account within the same region. 2. Create CloudFront Distribution for Videos We need the distribution to provide videos to users all over the world. To create one, please, follow the instructions: Visit the AWS CloudFront and press the button to create a CloudFront distribution home page As the origin, choose the output S3 bucket created in prev step For bucket access, choose to and then for the distribution. Also, don’t forget to hit the button. use OAI Create a new OAI update bucket policy Then, leave other options as default till you come to the section where you should choose option. Viewer Redirect HTTP to HTTPS Below the menu, you could set up an alternate CNAME and choose a certificate for your domain. This allows you to provide your URL to end-users. Then click the button and store the distribution’s id. Create distribution 3. Create Lambda function Then we need to create and deploy an API with 1 endpoint to upload videos. In the future, we could extend the functionality of the system to allow managing uploaded videos. We suggest using the Flask application with the Zappa package to set up and deploy the app on AWS Lambda. Flask is a lightweight and easy-to-use Python framework that fits our goals extremely well. As one option you could clone (by command git clone) the sample app here. In the next step, you need to configure the app to work with your infrastructure. In the configs folder, there is an file which should be populated with AWS IAM user’s credentials, input/output buckets, and CloudFront distribution id you’ve created early in this guide. app_config.json Next, install modules for Python. You could do this by following the instructions below: Create a virtual environment by using the command: python3 -m venv venv Activate the env: source venv/bin/activate Install packages pip install -r requirements.txt Add IAM user’s credentials to the file ~/.aws/credentials under video_uploader tag. It should look like this: Our next step is to set up Zappa to deploy the code on Amazon Web Services lambda. Type the command: Zappa init Choose the environment name and video_uploader profile Then, choose any bucket you want to upload code (it could be the default) and leave the modular path as is Press y to confirm settings After the setup, you could type to deploy the code to lambda. When the command is finished, you will get a URL to upload your videos to. You could test the functionality by executing this sample cURL: Zappa deploy dev curl --location --request POST '{your_api_url}/upload_video' \
--form 'video=@"{full_path_to_video}"' Replace your_api_url with the URL you got after Zappa deploy and full_path_to_video with a path to the video you want to upload. Summary VOD streaming is extremely useful in a variety of sectors and for a variety of purposes. Viewers may enjoy accessible media anytime and wherever they choose using VOD. Users get great material while also getting the convenience they want via . VOD streaming The nice thing about streaming and video on demand is that they’re not mutually exclusive: you can benefit the most from combining the two. You may use live broadcasts to connect with viewers and establish relationships, and VOD to reuse previously recorded content. All of this while lowering expenses and increasing service. By including live stream sessions in your video-on-demand library, viewers will have . They may watch available material whenever and wherever they choose thanks to a video on demand. Your target audience gets memorable online experiences as well as the convenience and performance that they expect. access to your content 24/7 The system is resistant to loads and simple to use. There are a lot of ways to improve such kind of API integration. But the most important will be the authorization functionality and the video managing panel. First published on here

Build a Video On Demand (VOD) Service with AWS Lambda, S3 and CloudFront

Comparing Cloudinary CDN and Amazon AWS CDN CloudFront 

I recently worked on implementing CloudFront for s3 bucket files. Most of the tutorials were doing that using console management(UI) but we at prefer “code as infrastructure”. So I had to do this using CloudFormation. With the help of a few StackOverflow links and tutorials, I was able to write CloudFormation Template. Blue Sky Analytics So What exactly CloudFront and CloudFromation are? These definitions will help you out. CloudFront: Amazon CloudFront is a fast content delivery network (CDN) service that securely delivers data, videos, applications, and APIs to customers globally with low latency, high transfer speeds, all within a developer-friendly environment. CloudFromation: On AWS, the CloudFormation service provides Infrastructure as Code capabilities. CloudFormation uses , configuration files defined in JSON or YAML syntax, that are human readable and can be easily edited, which you can use to define the resources you want to set up. CloudFormation reads a template and generates a , a set of resources ready to use on AWS. templates stack How is this gonna work? We are gonna use a private bucket, a public bucket can also be used but we don't use it to avoid direct access to bucket and leakage. CloudFront can access private bucket data using OAI(Origin Access Identity). An OAI is like a virtual user through which CloudFront can access private bucket. After creating OAI and using it in CloudFront, we need to update bucket policy, So that CloudFront with an OAI can access it. Resources we are going to build using CloudFormation: S3 Bucket OAI(Origin Access Identity). Update Bucket Policy CloudFront Distribution AWSTemplateFormatVersion: "2010-09-09" Resources: First, we need to define AWS template version and a "Resources" attribute to start writing resource templates. AWS template version defines the capabilities of a template. S3 Bucket Bucker resource template Bucket:
        Type: AWS::S3::Bucket
        Properties:
            AccessControl: Private
            BucketName: private-bucket
            Tags:
                - Key: description
                  Value: "Private files" "Bucket" is the name of resource template. We can use it any suitable name here. "Type" defines the type of resource, after this, we can define properties of resource. Each resource will have its own properties. OAI(Origin Access Identity) OAI resource template CloudFrontOriginIdentity:
    Type: AWS::CloudFront::CloudFrontOriginAccessIdentity
    Properties:
      CloudFrontOriginAccessIdentityConfig:
        Comment: 'origin identity' Update Bucket Policy Template resource to update bucket policy. BucketPolicy: Type: AWS::S3::BucketPolicy Properties: Bucket: private-bucket PolicyDocument: Version: '2012-10-17' Statement: - Effect: Allow Principal: AWS: !Sub 'arn:aws:iam::cloudfront:user/CloudFront Origin Access Identity ${CloudFrontOriginIdentity}' Action: 's3:GetObject' Resource: arn:aws:s3:::private-bucket/* Two of the most important fields, here are AWS field under principle and Resource fields. Recently created OAI can be used directly by using its template resource name like this "${CloudFrontOriginIdentity}" CloudFront Distribution Now the final CloudFront Distribution resource template. One property you need to be careful here is "Origins", that basically defines your bucket origin and "DomainName"  will be like this "${bucket name}.s3.${region}.amazonaws.com". publicDistribution: Type: AWS::CloudFront::Distribution Properties: DistributionConfig: Origins: - DomainName: private-bucket.s3.us-east-2.amazonaws.com Id: S3-private-bucket S3OriginConfig: OriginAccessIdentity: !Sub 'origin-access-identity/cloudfront/${CloudFrontOriginIdentity}' Enabled: 'true' Comment: Some comment DefaultCacheBehavior: AllowedMethods: - GET - HEAD TargetOriginId: S3-private-bucket ForwardedValues: QueryString: 'false' Cookies: Forward: none ViewerProtocolPolicy: redirect-to-https ViewerCertificate: CloudFrontDefaultCertificate: 'true' After completing CloudFromation template, you can use sam command to deploy and check status of its CloudFromation stack. Sam command : sam deploy --template-file [sample-template] --stack-name [Stack name] --s3-bucket sam-test-bucket --capabilities CAPABILITY_IAM Few links to learn more: https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-resource-cloudfront-distribution.html https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/quickref-cloudfront.html Thanks for reading 💓️

How To Configure CloudFront Using CloudFormation Template

Cómo configurar CloudFront usando la plantilla de CloudFormation

What is CloudFront? CloudFront is a CDN (Content Delivery Network). The CloudFront retrieves data from the Amazon S3 bucket and distributes it to multiple datacenter locations. The data that delivers through a network of data centers is called edge locations. The nearest edge location is routed when the user requests for data, so we will get some benefits like the lowest latency, low network traffic, fast access to data, etc. A picture will help you to clarify how this actually works, so take a look at the image shown below. We have stored our content or data in an S3 bucket located in a region in N.Virginia(us-east-1), and we have users located around the world who access that content. As the arrows show, whenever a user requests content, the request must go over the public internet to the source location–the S3 bucket in N.Virginia(us-east-1). Depending on the user’s location, content delivery can take a long time. The delays in the response or in content delivery might even cause some user requests to bounce and return an error from the page. Now imagine that we set up CloudFront in front of the S3 bucket. In the following image, you can see that there are no longer requests traversing the globe to get to our content. Instead, requests are routed to the ones that are closest in terms of delivery speed. These are also called “least latent” Edge Location. As shown with the green arrows, CloudFront then quickly serves cached content directly to the requesting user nearby. CloudFront retrieves the content from the S3 bucket origin if it is not cached with an edge server. Instead of the public internet, the content traverses the AWS private network and CloudFront optimizes the TCP handshake. Hence, the request and content return are still much faster than access across the public internet. So by using CloudFront will fetch data from CloudFront data centers instead of S3. The following Image is showing us the same thing. Let’s start the creation of CloudFront distribution. 1. Steps for creating a CloudFront distribution Sign in to the AWS Management Console and in the you can see a search box in that type cloud and choose . Find Services, CloudFront You should for the region at the top right. Global Click . Create Distribution Under click . Web Get Started For once you place the cursor in there you should see your available S3 buckets. Origin Domain Name Note: I already created S3 bucket in AWS so If you don’t have any website on S3 bucket then plz create it first Pick the website bucket you created. If it’s not listed type it in: e.g 2020–05–21-mybucket.s3.amazonaws.com  that is nothing but your bucket name. Leave blank. Origin Path The should have been pre-populated when you choose your bucket. Origin ID Click to . Yes Restrict Bucket Access Under select . Origin Access Identity Create a New Identity It will pre-populate the and append the bucket name. Comment For on Bucket choose options . [This will update the bucket policy for us]. Grant Read Permissions Update Bucket Policy Leave the blank. Origin Custom Headers For the section: Default Cache Behavior Settings Under select option . Viewer Protocol Policy Redirect HTTP to HTTPS For choose . Allowed HTTP Methods GET, HEAD Leave blank. Field-level Encryption Config Leave for . GET, HEAD (Cached by default) Cached HTTP Methods For leave it as the defaul Cache Based on Selected Request Headers t none (Improves Caching). For also leave it as the default . Object Caching Use Origin Cache Headers Under leave it as . Forward Cookies None (Improves Caching) Also for leave as Query String Forwarding and Caching None (Improves Caching). For select . Smoothing streaming No For select . Restrict Viewer Access (Use Signed URLs or Signed Cookies) No Leave as . Compress Objects Automatically No Leave as the default. Lambda Function Associations Scroll down to . Distribution Settings For leave the default . Price Class Use All Edge Locations (Best Performance) We will not be using WAF, so for leave it as . AWS WAF Web ACL, None Leave blank. Alternate Domain Names (CNAMEs) We will also use the for Default CloudFront Certificate SSL Certificate. For leave as . Supported HTTP Versions HTTP/2,HTTP/1.1,HTTP/1.0 Under type . Default Root Object index.html We can leave set to . Logging Off Leave checked. Enable IPv6 Finally set the to . Distribution State Enabled Click . Create Distribution Click on at the top left to see the status of CloudFront distribution being built or not. Distributions This can take 15–20 minutes to complete. 2. Restrict our S3 bucket policy to Cloud Front Click at the top left and type in S3 or select it from History. Services Click on your Bucket name . 2019-mm-dd-xx-mybucket IMPORTANT: Your bucket will have a different name. Click . Permissions Select . Bucket Policy We can see that CloudFront has added what we call an “Origin Access Identity” to the policy. {
"Version": " – – ",
"Statement": [
{
"Sid": "AddPerm",
"Effect": "Allow",
"Principal": "*",
"Action": "s3:GetObject",
"Resource": "arn:aws:s3::: – – -mybucket/*"
},
{
"Sid": " ",
"Effect": "Allow",
"Principal": {
"AWS": "arn:aws:iam::cloudfront:user/CloudFront Origin Access Identity E2V8GJ8FKJPGFQ"
},
"Action": "s3:GetObject",
"Resource": "arn:aws:s3::: – – -mybucket/*"
}
]
} 2012 10 17 2020 05 21 2 2020 05 21 Remove the public s3 section so it looks like following {
"Version": " – – ",
"Statement": [
{
"Sid": " ",
"Effect": "Allow",
"Principal": {
"AWS": "arn:aws:iam::cloudfront:user/CloudFront Origin Access Identity E2V8GJ8FKJPGFQ"
},
"Action": "s3:GetObject",
"Resource": "arn:aws:s3::: – – -mybucket/*"
}
]
} 2012 10 17 2 2020 05 21 This will only allow our specific CloudFront distribution access to our S3 bucket which is what we want. Click Save. 3. Steps for testing that we successfully locked down S3 from public view Browse to your S3 endpoint: Example: http://2020-05-21-mybucket.s3-website-us-east-1.amazonaws.com You will see a because we removed public access from the bucket policy. 403 Forbidden Click on the . (The blue hyperlink) CloudFront distribution ID Copy the URL under . Domain Name Browse to that URL and you should now see the index.html page. And your CloudFront is ready. Happy Coding 😃

Creating an Amazon CloudFront Distribution [A Step-by-Step Guide]

When developing an application it is important to have assets resized for different capabilities, screen sizes, and resolutions. This is important for many reasons. Low resolution images on high resolution displays make a bad UX High resolution images on low resolution displays waste both and device and server bandwidth resources Creating several thumbnails during image upload, can be a and for future redesigns. lot of work limits the agility Adopting an image in-app and it may be heavy for the app especially if you use Frameworks. cropping/resizing Resizing on the fly Instead of processing and resizing images into all necessary sizes upon upload, the approach of processing images on the fly has several upsides: Increased agility Reduced storage costs Resilience to failure In this post, we’ll be using a different approach and outline a method of lazily generating images, in which a resized asset is only created if a user requests that specific size. Creating a new bucket Open s3 console You can or use existing one but make sure the bucket’s read access permission is set to public. create new bucket Create new bucket, uncheck Block all public access Go to tab then choose permissions bucket policy To grant read-only permission to an anonymous user. Copy and paste this configuration to your bucket policy. Make sure to replace with the name of the bucket. You can read about bucket policy more in YOUR_BUCKET_NAME_HERE here { : , : [
        { : , : , : , : , : }
    ]
} "Version" "2012-10-17" "Statement" "Sid" "PublicRead" "Effect" "Allow" "Principal" "*" "Action" "s3:GetObject" "Resource" "arn:aws:s3:::YOUR_BUCKET_NAME_HERE/*" 6. Choose save Go to properties tab Choose static web hosting Choose use this bucket to host a website For , enter index.html index document Choose save. We’ll come later and add but for now just leave it empty. redirection rule Note the website’s endpoint* Creating CloudFront distribution for SSL Open CloudFront console Choose create distribution Choose web distribution Enter the that was noted previously, for static website’s endpoint origin domain name For viewer protocol policy — choose Redirect HTTP to HTTPS For object chaching set and TTL to 0 and TTL to 31536000 choose customize minimum default maximum Rest of the configuration you can leave as it is. You can now choose create distribution This can take few mins, wait until the status changes from ‘in progress’ to ‘deployed’. After the status is changed to deployed, note the CloudFront domain name. It should look similar to blabla .cloudfront.net Setting up permission for Lambda function Open IAM console Creating policy: In the IAM console select from the dashboard to the left Policies Choose create policy Navigate to tab JSON Replace with the name of the bucket that you’ve created and copy the following code into the policy document. Note that any leading spaces in your policy may cause a validation error. YOUR_BUCKET_NAME_HERE { : , : [
        { : , : [ , , ], : },
        { : , : , : }
    ]
} "Version" "2012-10-17" "Statement" "Effect" "Allow" "Action" "logs:CreateLogGroup" "logs:CreateLogStream" "logs:PutLogEvents" "Resource" "arn:aws:logs:*:*:*" "Effect" "Allow" "Action" "s3:PutObject" "Resource" "arn:aws:s3:::__YOUR_BUCKET_NAME_HERE__/*" Choose review policy Enter your policy name You can use any name but just for simplicity we’ll use the name imageResizePolicy Create policy Creating role: In the IAM console select from the dashboard to the left Roles Select create role Choose in for a use case, Lamda next In permission section search for the that you just created and choose that, You can skip the tags section and choose imageResizePolicy next Next In review section, enter role name, you can use any name but for the sake of simplicity we’ll be using the name imageResizeRole Create role Creating Lambda function Open Lambda console In the lambda console choose create function For function name enter your function name, now of course you can choose any name for just to be safe use as your function name. So that it’ll be easy for you to follow along. imageResize Choose as a runtime. Again you can choose any other language but we’re going to choose Node. You don’t need to worry about it as this lamba function works completely independent from your application. The only purpose it’ll serve is to provide you with resized images on fly. Node.js 12.x Expand the permission dropdown and select . use existing role Select imageResizeRole Choose create function Now choose and select from the dropdown add trigger API Gateway For choose API type HTTP API To allow all users to invoke the API method, for , choose and then . Security Open Add Now back to the main function window, you’ll see section. function code For code entry type choose upload a zip file Download function.zip file from this and upload the zip file, from top right corner link choose save To configure your function environment variables add two variables: For key, and enter for value, enter the bucket name that you created above. BUCKET; For key, enter ; for enter the domain the you created above from CloudFront, similar to URL value blabla.cloudfront.net On the main function window, choose API gateway and scroll a bit till you see our function’s API endpoint. Note the full path of the endpoint, depending upon your personal configuration these might be slightly different. Setting up S3 redirection rule Ok we’re almost there, phew that was a lot of configurations. In s3 console, open the bucket that you created above. To to properties and expand static website hosting, edit redirection rules. Replace (similar to ) with the hostname that you noted above and copy the following into redirection rules configuration. YOUR_API_HOSTNAME_HERE blabla.execute-api.us-east-1.amazonaws.com Notice the path inside <ReplaceKeyPrefixWith> tag, it’s the same from the one we noted above in picture. If your path is different the update it with your path. <RoutingRules>
  <RoutingRule>
    <Condition>
      <KeyPrefixEquals/>
      <HttpErrorCodeReturnedEquals> </HttpErrorCodeReturnedEquals>
    </Condition>
    <Redirect>
      <Protocol>https</Protocol>
      <HostName>__YOUR_API_HOSTNAME_HERE</HostName>
      <ReplaceKeyPrefixWith>default/imageResize?key=</ReplaceKeyPrefixWith>
      <HttpRedirectCode> </HttpRedirectCode>
    </Redirect>
  </RoutingRule>
</RoutingRules> 404 307 Test image resizing Upload a test image into your bucket to for testing. Once uploaded, try to retrieve resized versions of the image using your CloudFront domain: https://YOUR_CLOUDFRONT_DOMAIN/300×300/{image_name.jpg} https://YOUR_CLOUDFRONT_DOMAIN/25×25/{image_name.jpg} https://YOUR_CLOUDFRONT_DOMAIN/500×500/{image_name.jpg} You should see a smaller version of the test photo. If not, choose Monitoring in your Lambda function and check CloudWatch Logs for troubleshooting. You can also refer to the GitHub repo for a working example that you can deploy to your account. serverless-image-resizing YOU MADE IT!!! 🚀🎉🎊 Previously published at https://medium.com/@lamamanjil/image-resize-on-fly-with-amazon-s3-aws-lambda-and-cloudfront-for-ssl-8ab046ccf79f

Image Resizing after Upload with Amazon S3, AWS Lambda and CloudFront for SSL

https://getstream.io/winds is a popular RSS and Podcast application powered by — a service that allows you to build news and activity feeds in hours instead of months. Winds is and the frontend is easy to install in a local environment or in the cloud — a task that we will cover in this tutorial. To ensure that you make it through the tutorial, make sure to complete all of the prerequisites. Winds Stream 100% open-source please Prerequisites 📚 As with any tutorial, there are some requirements that come with it. For this post, you’ll need to ensure that you have the following up and running, and ready to go prior to continuing on. If you decide to skip the requirements, you’ll likely get hung up at one spot — and we don’t want that to happen. An AWS account with IAM Access to and S3 CloudFront installed on your machine and configured AWS CLI Have Winds cloned from to a local directory GitHub Creating a Bucket 🗑 First things first, head on over to the on AWS and create a bucket. I’m going to name my bucket “winds-2–0-hosted”, however, feel free to name yours anything you’d like! Click next and continue through the steps. We’ll be changing up some of the settings, so be sure to follow along. 😀 S3 console Next, within the “Properties”, open the “Static Website Hosting” tab and select “Enable website”. Fill in for both the Index and Error documents. By specifying as the Error Document, we can allow react-router to handle routes outside of the root. index.html index.html Open the “Permissions” tab and select “Edit Policy”. Here, we’re going to modify the permissions of the bucket to ensure that users can see and load the contents of Winds. The bucket policy can be found below: Setting up CloudFront 💨 CloudFront is the Content Delivery Network (CDN) service offered by AWS. CDNs optimize for speed by pushing your content out to edge locations around the world, making it highly available around the globe. Because Winds is a rather demanding application and has a lot of resources to add, we’re going to take it to the next level with CloudFront. Let’s go ahead and setup CloudFront. Once set up, we’ll then point it to the S3 bucket that we created in the previous step. To get started, head over to the and click “Create Distribution”. You’ll be presented with the following screen. Choose “Get Started” under the “Web” section. CloudFront console In the drop-down named “Origin Domain Name”, select the S3 bucket that to previously created. In our case, it will be named “winds-2–0-hosted.s3.amazonaws.com”. The rest of the settings will be pre-populated with the correct values, so scroll to the bottom and click “Create Distribution”. Your distribution will take a couple of minutes to generate, so be patient. Once generated, click on the CloudFront distribution, followed by the “Distribution” tab. Once that has loaded, click on the “Edit” button and set the “Default Root Object” to and click the “Yes, edit” button at the bottom. index.html Now, click the ID of your newly created distribution to reach its settings page, then click on the “Error Pages” tab. Select “Create Custom Error Response”. Select Yes for a custom error response, set “/index.html” for the “Response Page Path” and “200: OK” for the “HTTP Response Code”. This custom error page in the CloudFront distribution is analogous to the Error Document on the S3 bucket. When finished, click Create. You’ll want to create a second error page as well, this time with the “HTTP Error Code” set to “404”, the “Response Page Path” path to “/index.html” (again), and the “HTTP Response Code” to “200: OK”. Once that is complete, head back into the settings and click on the “Behaviors” tab. Click “Create Behavior” and select “Redirect HTTP to HTTPS” to force all traffic to HTTPS, as well as “Allowed HTTP Methods” to “GET, HEAD, OPTIONS, PUT, POST, PATCH and DELETE”. Then click “Yes, edit” at the bottom. That’s it for now! You can find the URL on the distribution listings page, under the Domain Name column. Let your distribution sit for a couple of minutes to let all of the settings propagate. Change the API URL 👨‍🚀 I wanted to quickly note that it’s important to change your API URL as you’ve probably decided to your API elsewhere. To do so, head into the Winds directory and under there will be a .env file. Change the environment variable. Be sure to do this before uploading or your application will not load. /app REACT_APP_API_ENDPOINT Prepping the Deploy Script 🚀 When deploying Winds on our end, we use a handful of scripts for both the front and backend. Luckily, the frontend only requires one file. This file must be located inside of your Winds directory and have the proper permissions. To create the script, move into the Winds directory and create a file called . Open and paste the following into the file: deploy.sh deploy.sh Next, let’s set the permissions on the file: Note: You must change the following in order for the deploy script to properly work! 1. On line 16, change the homepage to the Domain Name provided by CloudFront as shown in the screenshot below: 2. On line 20, change the S3 URL to your created S3 bucket URL (e.g. winds-2–0-hosted): 3. On line 23, change the S3 URLs to match your S3 bucket URL (e.g. winds-2–0-hosted) in the two specified locations: 4. Last, on line 25 update your Distribution ID (provided by CloudFront and shown in the following screenshot): Once the file is properly set up, it will handle the following for you: deploy.sh Verify that you want to actually deploy Move into the /app directory Create a build directory with compiled files and assets Sync the build files and assets to your S3 bucket Sync non-build, but necessary files to S3 Deploying to S3 & CloudFront 🤟 Now that everything is created and properly set up, let’s go ahead and give it a shot! Make sure that you are in the Winds directory, and type to fire off the script. When it prompts you, press “y” followed by pressing enter. A series of events will happen automatically and you’ll see the output below when the script is complete. ./deploy.sh You did it! Your application should now be live at the URL that CloudFront provides. Final Thoughts 🤔 If you’d like to change the URL, I’d recommend taking a look at AWS Route53 for a DNS service. You can map your custom URL to your CloudFront distribution making Winds have a custom domain! If you haven’t had a chance to check out the post on how to deploy the Winds API, you can read it . As always, if you have any thoughts or comments, please leave them in the comments below! here Happy coding! 🎉 Originally published at getstream.io on September 6, 2018.

Deploying the Winds App to Amazon S3 and CloudFront

Configuring CloudFront to support push-state URLs This one seems like an easy one, however I’m appalled by how much time can be wasted configuring for static website hosting and there must be a better way: AWS — : we will be hosting static assets so no messing around with servers for that — : by leveraging AWS CDN, we’ll get world-class global distribution and caching for pennies — : this is not a tutorial of the AWS console. Some of the marketing agencies I consult for would typically deploy new landing pages or static websites every week. If that’s something you do regularly too, it is worth investing an hour into automating this into a push-button procedure isn’t it? serverless cheap and ultra-performant automated About automating There is something very powerful about automating even the little things. The benefits are much more than the sum of time savings over time, it will reduce human errors, improve the reliability of your service, get your employees busy with higher-value work and boost employee morale by eliminating boring low-value tasks. It improves your business globally. A game changer: with Lambda@Edge, we can now configure CloudFront to support push-state URLs The problem so far with hosting an SPA on CloudFront is that we couldn’t support because CloudFront doesn’t offer a “catch-all” configuration i.e. we couldn’t serve the same file for any request to the domain. The solution was to use hash-state “ugly” urls which was a turn-off for most. push-state URLs index.html In 2017, AWS released Lambda@Edge, a service that runs your Lambda functions “at the edge” of the cloud i.e immediately before/after the CDN queries its cache: : after CloudFront receives a request from a viewer viewer request : before CloudFront forwards the request to the origin origin request after CloudFront receives the response from the origin origin response: : before CloudFront forwards the response to the viewer viewer response This will let us rewrite the requests on so that we always serve for all push-state routes requests. Happy days! origin request index.html What do I need? an existing domain name registered as a hosted zone in your AWS account a wildcard SSL certificate for all subdomains and the CLI installed and configured on your machine terraform aws a dummy React or Vue application to deploy Show me the Terraform module! I’ve released a module that does all the heavy lifting for you: Terraform _terraform-aws-single-page-application - Terraform module which lets you deploy SPAs on AWS using Git Push_github.com li0nel/terraform-aws-single-page-application Copy the example files into your SPA root folder: wget https://raw.githubusercontent.com/li0nel/terraform-aws-single-page-application/master/examples/main.tf   wget https://raw.githubusercontent.com/li0nel/terraform-aws-single-page-application/master/examples/outputs.tf   wget https://raw.githubusercontent.com/li0nel/terraform-aws-single-page-application/master/examples/terraform.tfvars wget https://raw.githubusercontent.com/li0nel/terraform-aws-single-page-application/master/examples/variables.tf Replace the values in with your CLI profile name, your app name and your domain name. Make sure you have an existing certificate for in the region. terraform.tfvars aws *.cdn.mydomain.com us-east-1 Run then and wait for the S3 bucket, Lambda function and CloudFront distribution to be created: terraform init terraform apply You can now compile your front-end assets locally and push them directly to the S3 bucket to deploy your application: yarn build && aws s3 cp --recursive --acl=public-read build/ s3://$(terraform output s3_bucket) An alternative is to use the CodeCommit Git repository and CodePipeline pipeline that has been created by the Terraform module to let AWS build your application, run your tests and deploy on S3. Either way, your application is now running on CloudFront! The push-state URL working properly when refreshing the page Lionel is Chief Technology Officer of London-based startup Wi5 and author of the Future-Proof Engineering Culture course . You can reach out to him on https://getlionel.com

How to host a Single Page Application with AWS CloudFront and Lambda@Edge

AWS S3 . It is cheap, scalable, and “performant”. Especially when it tag team with CloudFront. Static Website Hosting This is a documentation of how to host a Single Page Application (React for this case) on AWS S3 with SSL over CloudFront using this of mine as an example. pet project 1) The project A simple static site so no redux is used; this setup would also work with redux. So its gonna be react and react router mainly. Here are the specifics: react: ^15.6.1 react-router: ^4.1.2 The bundler I am using is . webpack: ^3.5.5 2) AWS S3 S3 can host static website apart from just storage. that each bucket is meant for only 1 website, that is you cannot have a bucket called and have each directory hosting 1 website. No. It is going to be per website per bucket. Note my-static-websites Set up the static website hosting configuration as such for the bucket. Take note of the . Endpoint This setup is saying: When users visit the root path of my website, show them the file . index.html When users visit the a page that does not exist, show them the default S3 error message on their browser. So when we upload the react project into the bucket: At the root path, users can see the site up and alive!!! Users can also navigate to different paths!!! hitting refresh when the path is instead of will show you a blank screen, or the page if one was setup :(((( But /something / error.html What is happening? Well the path is looking for a file in the S3 bucket but it was not to be found. Since this is a Single Page Application, there is only 1 html file, 1 GOD html file. /something something.html So here is the challenge. We need to map paths to the file. all index.html Since this is a react project, we do not need to map each path to a specific other html page like a typical website; the will load the javascript bundle and react router will get to work to show users the correct page based on the path. index.html Hygiene pages Not sure if this is the correct term for and files but yea you’ll need these files for . These files go into the root directory of your bucket as siblings to the file. and the url to them are, eventually, and respectively. sitemap.xml robots.txt SEO index.html https://www.yourdomain.com/robots.txt https://www.yourdomain.com/sitemap.xml 3a) AWS CloudFront — Distribution CloudFront is the CDN of AWS it can handle the mapping of the routes, on top of caching the site. Start off by creating a distribution. The key configurations I will like to mention are: web Origin Domain Name — Upon focusing on this field, there will be a dropdown listing all the buckets that you have in your AWS S3. Do use any option from this list. Instead, enter the domain of the of the S3 bucket as mentioned in the previous section here. NOT Endpoint static-website-hosting-enabled Viewer Protocol Policy — Select to ensure your website is always viewed over HTTPS, and there is no duplicate instance under the HTTP protocol that is accessible by the public. Redirect HTTP to HTTPS Cache Based on Selected Request Headers — Select and add in the header. This is to avoid any CORS related errors. Whitelist Origin Alternate Domain Names (CNAMEs) — enter the non-www and the www domain name here, or any other subdomain you have may have intended, separated by a line break or comma. SSL Certificate — Select and upload you own ssl certificate, along with the private key and CA bundle via . Custom SSL Certificate Amazon Certificate Manager Compress Objects Automatically — Select . your uncompressed assets from S3 and improved your page speed, by . Exchange all the deep apache/nginx/IIS setups with just a radio button — that’s like . Yes CloudFront will automatically compress Google standards trading a wight for a dragon Create the CloudFront distribution and wait for it to get deployed. Take note of the distribution’s Domain Name . 3b) AWS CloudFront — Error Pages After creating the CloudFront distribution, while its status is , proceed to the tab. Handle response codes 404 and 403 with . In Progress Error Pages Customize Error Response 1 week or 604800 seconds of caching. Google recommends What we are doing here is to set up CloudFront to handle missing html pages, which typically occurs when a user enters an invalid path or, in particular, when they refresh a path other than the root path. When that happens: CloudFront will be looking for a file that does not exist in the S3 bucket; there is only 1 html file in the bucket and that is the for the case of a Single Page Application like this project example index.html A 404 response will be returned and our custom error response setup will hijack it. We will return a 200 response code and the page instead. index.html React router, which will be loaded along with the index.html file, will look at the url and render the correct page instead of the root path. This page will be cache for the duration of the TTL for all requests to the queried path. Why do we need to handle 403 as well? It is because this response code, instead of 404, is returned by Amazon S3 for assets that are not present. For instance, a url of will be looking for a file called (without extension) that does not exist. https://yourdomain.com/somewhere somewhere PS. It used to be returning 404, but it seems to be returning 403 now; either way it is best to handle both response codes). 4) DNS I intend to use the www version of the domain. Go to the DNS zone file and set up as such. This setup indicates: will be redirected to domain.com www.domain.com requests will be rewritten, if valid, from to http https I am using as my DNS service provider, and they come with an option to redirect or to at the DNS level. namecheap.com https http non-www https www However. If your DNS service provider does not provide this function, you can use AWS S3 to do the redirect instead. Create bucket with these settings. another Set the value DNS A record of the root domain to the end point of this bucket. What will be achieved is all request will be directed to this bucket. This bucket will in turn redirect the request to the domain, which points to the bucket where the files are. And yes it will be a . In case you are wondering, . non-www www 301 redirect this is the significance of a 301 redirect Conversion of to will be handled by CloudFront configuration (Viewer Protocol Policy) that was setup previously. http https At this point of time, you should be able to access your site like a normal website. Refreshing at a path other than the root path should also work. All non https requests will be redirected under the https protocol. All non www request will be redirected to the www domain under the https protocol as well. Bots and crawlers should be able to access your and files as usual. robots.txt sitemap.xml 5) Conclusion Pros Financially friendly. You basically pay for what you only use, so you would not be wasting any penny on under utilized resources that comes with a monthly payment model. On top of that, this Cloudfront & S3 combination also saves you some money because it is a lot cheaper to transfer data out to the Internet via CloudFront than S3. Not to mention the better performance of a CDN. Scalability friendly. If somehow your site gets really popular, there will not be a scalability issue due to the surge in traffic because AWS CloudFront will be taking care of that for you. There is no need for any upgrade of plans with other hosting companies. Performance friendly. Since this whole site is sitting on top of a CDN, delivery of the site and the assets are going to be super fast. DDoS unfriendly. Since the site is behind AWS CloudFront, is, once again, handled by CloudFront. DDoS attack are guarded against by Amazon’s own technology and I will place my bets on their cyber security technology and reliability than on other hosting companies. attack against DDoS Security friendly — Since CloudFront is now handling the SSL configurations, you will see that the SSL tests for your domains are A grade on . SSLLabs Cons This works only on static sites. It will take a humongous amount of traffic to even slow down a static website substantially. Most of the bottle necks in a typical application is when it interacts with a backend that involves logic computation and database queries. Since the site is cached on the CDN, any changes will not be seen immediately and have to wait until the cache expires. This is something that comes along with any caching mechanism. We can mitigate it by invalidating cache (which will incur charges). If your javascript file names are hashed, then you can ignore the the javascript files and just need to invalidate the file. Alternatively, you can give a lower caching period for the file. index.html only index.html Every Single Page Application’s bug bear is the requirement for server-side rendering. Bots and crawlers are not able to get the meta data of the site because they do not allow javascript to execute, apart from . So if your site is only concerned about SEO on Google, this setup is good to go. But if you are reliant on other search engines, or if you are marketing the site via social media like Facebook, this is not ideal. (TODO serve html pages using API Gateway and Lambda) Googlebot As all 404 and 403 responses are hijacked to return 200, you will probably not receive any 404 errors on Google Search Console (GSC), if you had indexed your website there. These 404 reports provided by GSC are useful to tell you which pages are having error and will notify you about it. Without them you will not know which pages are down or if there are any broken links linking to other parts of your website. Side quest In this section of the article, I will be documenting how to automate the deployment process of such a site in such a setup from just the command line. 1) AWS IAM To start off, you will need to create an IAM user and give it the necessary S3 permissions. Note the and the , as well as the . access key id secret access key User ARN IAM users are access control configuration in your AWS account, principally to answer the question of who can do what to which of the services under your account. Let’s call this user . iam_user 2) AWS S3 Change the bucket policy to allow this to make changes to the bucket. iam_user {"Version": "2012-10-17","Id": "someID","Statement": [{"Effect": "Allow","Principal": {"AWS": "arn:aws:iam::123456789:user/iam_user"},"Action": "s3:*","Resource": "arn:aws:s3:::bucket-name"}]} 3) Deployment As this is a simple, mostly static, website, there is no testing scripts or any CI server set up for the deployment procedure. It will just be a simple task to upload new files to the correct bucket in S3 using . AWS CLI Cleanup But before uploading, make sure you clean up the distribution folder where you build your files for the production environment. Since I use as my bundler, I utilise the to help me dispose of old files before building new ones. This is to prevent uploading the same old assets again to the bucket. webpack clean-webpack-plugin # webpack.config const CleanWebpackPlugin = require('clean-webpack-plugin')const HtmlWebpackPlugin = require('html-webpack-plugin')const pathsToClean = ["dist"]const cleanOptions = {} ... output: {path: path.resolve(__dirname, "dist", "assets"), // all files are bundled into the dist/assets sub-directorypublicPath: '/assets/',filename: 'bundle.js'}, ... plugins: [...,new CleanWebpackPlugin(pathsToClean, cleanOptions), // cleanup the whole "dist" foldernew HtmlWebpackPlugin({template: "./src/index.production.html",filename: "../index.html" // all files are bundled into the dist/assets sub-directory, but index.html will be placed 1 directory up in the dist directory itself}), ...] Uploading Now to upload the files to S3. To prevent any Tom Dick and Harry from being able to do so, authentication is required. This is where all the work for IAM comes into play. We will use a script to do the uploading, with custom configuration to authenticate the request. You can use flag to test your script before actually doing the upload. This is the final version of my script. --dryrun aws s3 cp ./dist s3://better-cover-letter  --recursive --exclude "*.DS_Store" --acl public-read --cache-control public,max-age=604800 --dryrun --profile iam_user The flag is to prevent the upload of the irritating, ever present file in macOS. --exclude .DS_Store The flag will set the access control level of the files. Make it public readable so people can access your site, otherwise they will be slapped with a message. --acl 403 Forbidden The flag adds the cache-control header to the S3 objects when Cloudfront calls for them. These cache control headers will be passed to the browser to and thereby increasing page speed. 604800 is 1 week in seconds, so this value will cache these assets for a week. --cache-control leverage on browser caching max-age [Google] recommend[s] a minimum cache time of one week and preferably up to one year for static assets, or assets that change infrequently The flag is used to set the specific IAM user credential to authenticate this operation. As I am using this same macbook pro for my work and my personal projects, I have multiple AWS accounts to handle, thus the need for this flag to differentiate the different IAM users. Check out for more information. These are my config and credentials files for your reference. --profile AWS CLI named profiles # ~/.aws/config[default]region=us-west-2output=text # ~/.aws/credentials[iam_user]aws_access_key_id=somethingaws_secret_access_key=something [company_user]aws_access_key_id=something_elseaws_secret_access_key=something_else The and are specific to the that was created. aws_access_key_id aws_secret_access_key iam_user Once you are ready, you can remove the flag and do a test run to ensure that your files are indeed uploaded to the correct bucket. Yes, a run. It is not the end of the deployment step. We can go further to completely automate the whole process. --dryrun test : AWS S3 does not charge data transfer in to the bucket, only out. So feel free do spam deployment. (In fact, S3 does not charge data transfer out to Cloudfront.) NOTE Combine the Steps As it stands now, we have to build our site first using to generate the files, then upload the files using the command. webpack -p — config webpack.config.js aws s3 cp To make our life better, we can create a new script command to run these commands one after another, without having us to be there waiting for the first command to finish then manually execute the other. # package.json ..."scripts": {..."deploy": "webpack -p --config webpack.config.prod.js && aws s3 cp ./dist s3://better-cover-letter  --recursive --exclude "*.DS_Store" --cache-control public,max-age=604800 --dryrun --profile iam_user"...} So just run and these will happen in chronological order. npm run deploy Old production files are cleaned up by clean-webpack-plugin New production files are compiled into the folder (based on my webpack config file) dist The production files are then uploaded to S3 and ready for access once the cache in the AWS Cloudfront CDN expires. There it is, the fully automated process for uploading the static website. More Housekeeping (Optional) If you are bundling your javascript files with a hash like me, you will find your S3 bucket accumulating with old js file instead of getting replaced by the new ones since they are different files by virtue of the hash in their file name, eg . Not so much if you are uploading just which will replace any present in the bucket. bundle-0af19d01880334b789.js bundle.js bundle.js Since storing files in S3 isn’t free, albeit not that expensive either, its still wise to remove files that you will never be using again. So we can use AWS CLI again to do a removal of these old js files before upload (note: I am leaving the files in the root directory of the bucket untouched, just cleaning up the folder). assets aws s3 rm s3://better-cover-letter/assets --recursive --profile iam_user --dryrun Once again, combine them in the deploy script.

Hosting Static React Websites on AWS S3 (& CloudFront) with SSL

This article originally appeared on . the Cosmic JS Blog So you’ve read all the great reasons to , and you’ve produced a great looking single page app to display your content. But how do you serve it to the user? go API-first You COULD set up a full Node server to host, route and serve your site. Or you could just host your website statically using AWS S3 and CloudFront! This means you simply put your website source files online, and they’re served immediately to the user, no server processing needed. There are several big advantages to static hosting with client-side rendering: + Blazing Fast (no server preprocessing) + Really simply (no server architecture to maintain) + Crazy Available (your server can’t break if there isn’t one) + Instant Updates (don’t have to wait for the site to rebuild to see your changes) Setting up a static site is easier than you might think, and this tutorial walks you through how to set up a static website that gets its data from Cosmic JS and renders on the client in under 10 minutes! We’ll also cover some tips and tricks on getting the most out of your static site that you might not have known, which will be useful, even if you’re not building a purely static site. TL;DR View the demo here Check out the React GraphQL Static Site codebase The site we’ll be hosting Our example website is a static single page app, in our case using and , with handling routing, and to fetch data (tutorial coming soon). webpack react react-router Cosmic JS's GraphQL API If you’d like to follow along at home, you can . download our demo code here Our demo site is structured as follows: .├── asset-manifest.json├── favicon.ico├── index.html└── static└── js├── main.3c8c3c1e.js├── manifest.4a707e54.js├── vendor.94287fcf.js├── vendorConstant.0e34bc16.js└── vendorReactApollo.c8e28b23.js We’ve broken our into several different files so that they can be cached independently; if is the only file that changes, it's the only file that our clients will have to re-get when they revisit our site. js vendorReactApollo is the only HTML file we need to deploy, and it's minimal: index.html <!DOCTYPE html><html lang="en"><head><meta charset="utf-8"><meta name="viewport" content="width=device-width,initial-scale=1,shrink-to-fit=no"><link rel="shortcut icon" href="/favicon.ico"><title>React App</title></head><body><div id="root"> </div><script type="text/javascript" src="/static/js/manifest.4a707e54.js"> </script><script type="text/javascript" src="/static/js/vendorConstant.0e34bc16.js"></script><script type="text/javascript" src="/static/js/vendorReactApollo.c8e28b23.js"></script><script type="text/javascript" src="/static/js/vendor.94287fcf.js"></script><script type="text/javascript" src="/static/js/main.3c8c3c1e.js"></script></body></html> With this single page app structure, we’re ready to start serving it as a static site using AWS S3! S3 setup AWS S3 buckets can serve their contents as static web pages with little setup. This provides a hassle free solution for serving static files, like our website, quickly and simply. Enabling website hosting on S3 requires a few steps: 1. Bucket Policy First, you need to set permissions for all files in your S3 bucket so they can be read by anyone. You can copy-paste the code we’ve used, and replace our bucket name ( ) with your own: codogo-cosmic-js-demo-blog {"Version": "2012-10-17","Statement": [{"Sid": "AddPerm","Effect": "Allow","Principal": "*","Action": "s3:GetObject","Resource": "arn:aws:s3:::codogo-cosmic-js-demo-blog/*"}]} 2. Website Hosting Click on , and fill in the details as below. Static website hosting The is required, and tells the S3 bucket which file to serve if we navigate to the root of our website. Index Document The is the file that’s served whenever a client navigates to a file that doesn’t exist in the bucket. This is the most important setting for this tutorial, as it means no matter which path the user navigates to, they are still served our file. Error Document index.html 3. Upload your files Upload your site to the root of your S3 bucket and your site will be ready to serve. Testing the hosting out Navigate to the URL shown in the box, and you should see your web app! You can . Endpoint Static Website Hosting check out our demo here Try navigating to other pages and refreshing the page — you should see that your web app loads as if the pages you’re navigating to actually exist on the server. Next steps For a small site with low, unimportant traffic — for example, a company internal website — this hosting would be sufficient. However, there are two main issues with this hosting solution: : The website will only be served from the AWS region where your bucket lives, so people in other parts of the world will see poor performance. Served from one region : If you open your browser network inspector and navigate to a page, you’ll see that the server is responding with a code, instead of the / that we want. HTTP 404 Response code 404 200 302 Having every page of our site return a code is terrible for SEO as any web-crawler that checks the HTTP response code will think every link on our site is broken! In the next section we'll look at how we can use AWS CloudFront to cache our site, handle our traffic, and fix our error code issue. 404 CloudFront Config 1. Create A CloudFront Distribution We need to create a CloudFront web distribution through which all our web traffic will flow. This will handle the world wide distribution of our S3 website, and can intercept and correct our codes. 404 In the field you'll be given the option to pick your S3 bucket. **Do Not Do This!**Instead, set the to be your S3 website URL. For example, for our demo it would be codogo-cosmic-js-demo-blog.s3-website-eu-west-1.amazonaws.com, not codogo-cosmic-js-demo-blog.s3.amazonaws.com. Origin Domain Name Origin Domain Name 2. Cache long term files Now we’re going to tell CloudFront how to cache our files. To do this we need to create a new Behaviour. static We want our to match everything in our static folder. The should be left as they are. Path Pattern Object Settings 3. Correct Response Code Now we create a new error page to intercept the errors from S3. Fill out the custom response as above. You’ll need to give your CloudFront distribution around 15 minutes to fully deploy, but then it should be accessible at it’s domain name, ours looks like this: d2hqptkles3gsg.cloudfront.net. Conclusion And there you have it: A simple client-side rendered static site to display content from Cosmic JS that’s super fast and super simple. If you’re making a static site, or anything else, with Cosmic JS or ; we’d love to see what you’re making. This post was written by , a fresh new digital agency with a passion for creating amazing digital experiences. Keep an eye out for our next post, on best practice for making a Cosmic JS site using React and GraphQL. get in touch on out Slack Twitter Codogo

How To Build a React GraphQL Static Site Served From AWS CloudFront

For many years Nginx has been a default solution to serve as a reverse proxy for Rails applications. However, with the release of Kamal, the Rails community opened Traefik as a new reverse proxy solution. Within my 15 years of experience with Rails, I created an almost perfect configuration for Nginx that migrated through all my projects. With Traefik, I had to start from scratch. Traefik Intro Traefik is an Edge Router; it means that it’s the door to your platform and that it intercepts and routes every incoming request but does not process requests itself. When an incoming request comes to Traefik, it is getting served through the EntryPoint. Traefik can listen to multiple EntryPoints, and each EntryPoint is associated with a port and a protocol (HTTP/HTTPS, TCP, UDP). After the request is received, it is passed to the Router. Router is a rule that tells Traefik how to handle the request. It can be based on the path, the host, the headers, etc. The router will then forward the request to the appropriate service. But before a request is forwarded, it can be modified by the Middleware. Middleware is a piece of code that can modify the request or the response before it reaches the service. Middleware is the place where you can add authentication, rewrite the path, add headers, etc. To better understand how Traefik works, I provide an image from the official . Traefik documentation Compared to Nginx, the Traefik configuration is entirely different. Besides a traditional configuration file (which can be both in YAML and TOML formats), Traefik can be configured with dynamic configuration. It means that you can configure Traefik with a file, but also with a REST API, a Docker label, a Kubernetes annotation, etc. Since Kamal is a Docker orchestration tool, Traefik configuration is done with Docker labels. It means that you don’t need to write a configuration file; instead, you should provide labels to your Docker containers. Labels work dynamically, so you don’t need to reboot Traefik to apply them. At the same time, operating a Docker label can be pretty tricky, especially at the beginning. Traefik Dashboard The good news is that Traefik works pretty good with Kamal out of the box with Kamal (Kamal provides basic default configuration, which I will show later), and maybe you won’t need to change anything. However, if you want to take a look at how Traefik is configured, you may want to enable a . web dashboard To do this, open your Kamal config (usually ), and add the following lines. config/deploy.yml # config/deploy.yml

traefik:
  args:
    api.dashboard: true
  labels:
    traefik.http.routers.dashboard.servixce: api@internal
    traefik.http.routers.dashboard.rule: "Host(`traefik.onetribe.com`)"
    traefik.http.middlewares.auth.basicauth.users: user:$2y$05$us7dsDg56EJ/qojvhLvy9OEshrzoWjGSKXziBqeiFq3Ehf1pAiGSG
    traefik.http.routers.dashboard.middlewares: auth@docker If you have a separate configuration for production and staging environments, you probably will have the last two labels in your environment-specific configuration files. # config/deploy.production.yml

traefik:
  labels:
    traefik.http.routers.dashboard.rule: "Host(`traefik.onetribe.team`)"
    traefik.http.middlewares.auth.basicauth.users: user-production:$2y$05$us7dsDg56EJ/qojvhLvy9OEshrzoWjGSKXziBqeiFq3Ehf1pAiGSG

# config/deploy.staging.yml

traefik:
  labels:
    traefik.http.routers.dashboard.rule: "Host(`traefik.staging.onetribe.team`)"
    traefik.http.middlewares.auth.basicauth.users: user-staging:$2y$05$us7dsDg56EJ/qojvhLvy9OEshrzoWjGSKXziBqeiFq3Ehf1pAiGSG Let’s go line by line in file. config/deploy.yml On line enables Traefik Dashboard, as it is disabled by default. 5, api.dashboard: true On line I defined a router that will serve the dashboard. The internal Traefik API service is called ; I attached it to the router. 7, api@internal dashboard Then, on line I defined a rule that tells Traefik to serve the dashboard on the domain. Rule can be based . 9, traefik.onetribe.team on the path, the host, the headers, etc Since I don’t want to expose the dashboard to the public, I added a basic authentication middleware on lines and . Traefik supports different authentication methods: , , , etc. I used basic authentication, as it is the easiest to configure. 9 10 basic digest forward Password can be encrypted with the command, which is a , and this is really amazing how software, which is 20 years old, is still in use. To encrypt a password for a new user, run the following command. htpasswd part of the Apache toolkit htpasswd -nB user Traefik Logging Besides the visualization of the configuration, it is important to have good logging. Most of my Kamal projects use AWS CloudWatch for logging, so I configured Traefik to send logs to CloudWatch. To do this, I added the following lines to the file. config/deploy.yml logging:
  driver: awslogs
  options:
    awslogs-region: us-east-1
    awslogs-group: application
    awslogs-create-group: true
    tag: "-" This will configure logging for your application containers. In the example above, I used as a log group; you can use any name you want. The option is important; I used the container name and ID to distinguish logs from different containers. The picture below demonstrates how logs are organized in the CloudWatch. application tag As you see, besides application container logs, I also have a Traefik log stream, which includes both error and access logs (which is not the best, but it is how it is). Traefik logging can be configured by passing arguments to the Traefik container. I assume that my health check requests are pretty fast, so I set the minimum duration to 50ms to filter not important data. traefik:
  args:
    log.format: json
    accesslog: true
    accesslog.format: json
    accesslog.filters.minduration: 50ms Middlewares 101 I already talked about middlewares earlier in this article; let’s have a closer look at them. Middlewares are a piece of code that can modify the request or the response before it reaches the service. I used a basic authentication middleware to protect the Traefik dashboard. As we already know, each router has a set of settings and is also a setting that accepts a list of middlewares. Each middleware should be defined with it is own labels. middlewares traefik.http.middlewares.<middleware_1>.basicauth.users: username:<encrypted_password>
traefik.http.middlewares.<middleware_2>.ipwhitelist.sourcerange: 127.0.0.1/32
traefik.http.routers.<router_name>.middlewares: <middleware_1>, <middleware_2>, <middleware_3> : Traefik middlewares are not global. They are attached to the router, so you need to define them for each router separately. Important Kamal adds a to each service that uses Traefik. It is essential to know because if you add a middleware to a router, it will replace the default middleware. If you want to add a middleware to the default middleware, you must add it to the list of middlewares. default middleware WWW Redirect Redirecting from a www to a non-www domain is a common practice. It is important to have a single domain for SEO and security reasons. In Nginx, I used to do this with a simple directive. rewrite server {
  server_name www.onetribe.team;
  return 301 $scheme://onetribe.team$request_uri;
} To do this with Traefik, you need to configure on the router. RedirectRegex middleware servers:
  web:
    hosts:
      - <ip_address>
    labels:
      traefik.http.routers.onetribe-web.middlewares: onetribe-web-www-redirect@docker
      traefik.http.middlewares.onetribe-web-www-redirect.redirectregex.regex: ^http://www.(.*)
      traefik.http.middlewares.onetribe-web-www-redirect.redirectregex.replacement: https://$1 You probably noticed that I used schema in regex. Why not ? In all my configurations with Traefik, I do HTTPS termination on the load balancer before the request reaches Traefik. In Rails, 7.1 introduced to handle such configurations. http:// https:// config.assume_ssl option Compress Almost all Nginx configurations I saw had a directive. Compressing responses to save bandwidth and speed up the website is a good practice. Digital Ocean has an , explaining how HTTP compression works and how to configure it with Nginx. gzip excellent tutorial server {
  # ...
  gzip on;
  gzip_comp_level 2;
  gzip_min_length 1000;
  gzip_proxied any;
  gzip_types text/plain text/css application/json application/javascript text/xml application/xml application/xml+rss text/javascript;
  gzip_vary on;
  # ...
} Traefik has a that can be used to achieve the same result. Compress middleware servers:
  web:
    hosts:
      - <ip_address>
    labels:
      traefik.http.routers.onetribe-web.middlewares: onetribe-web-compress@docker
      traefik.http.middlewares.onetribe-web-compress.compress: true Full Example The example below combines two middleware defined above and a middleware that is added by default by Kamal–Retry. The Retry middleware reissues requests a given number of times to a backend server if that server does not reply. servers:
  web:
    hosts:
      - <ip_address>
    labels:
      traefik.http.routers.onetribe-web.middlewares: onetribe-web-www-redirect@docker,onetribe-web-retry@docker,onetribe-web-compress@docker
      traefik.http.routers.onetribe-web.rule: PathPrefix(`/`)
      traefik.http.services.onetribe-web.loadbalancer.server.scheme: http

      traefik.http.middlewares.onetribe-web-www-redirect.redirectregex.regex: ^http://www.(.*)
      traefik.http.middlewares.onetribe-web-www-redirect.redirectregex.replacement: https://$1

      traefik.http.middlewares.onetribe-web-compress.compress: true

      traefik.http.middlewares.onetribe-web-retry.retry.attempts: 5
      traefik.http.middlewares.onetribe-web-retry.retry.initialinterval: 500ms Rails Assets Handling Last, it is not about the Traefik but about the Rails configuration. With Nginx, I usually didn’t care about how my CSS, JS, and other static assets were handled; Nginx did this for me. All required headers (like compression that we discussed above and cache control) were added by Nginx. Traefik is not a web server; it is an edge router, so it doesn’t handle static assets. It means that you need to configure Rails to handle static assets properly. Stop, Rails static file handling is slow and not efficient. And you are right, or at least you were right. Modern versions of Puma handle static assets well. Besides, in 2024, we usually use cloud storage like AWS S3 to handle user uploads, so Rails will only serve CSS and JS files, which is not a big deal for Puma. I ended with something like this in my file. config/environments/production.rb # config/environments/production.rb
  # ...

  config.public_file_server.enabled = true
  config.public_file_server.headers = {
    "Cache-Control" => "public, max-age=#{30.days.to_i}, must-revalidate"
  } header is important; it tells the browser how long it should cache the file. In this example, I set it to 30 days. Must-revalidate tells the browser to revalidate the file after the cache expires. Cache-Control With such settings, Rails will serve static files only once for each client. If you think this is not enough, you can use a CDN like Cloudflare or AWS CloudFront to cache responses from Rails and dramatically reduce the load on your server. Conclusion As it usually happens with new technologies, it takes time to get used to them. Traefik is not an exception. It took me a while to understand how it works and how to configure it. At the same time, I see the benefits of using Traefik over Nginx with Rails – the main one is, of course, support of Docker out of the box. I believe this was the main reason why Basecamp decided to go with Traefik. Almost a year has passed since I switched our first project to Traefik, and I’m happy with the result. I hope you will find this article useful. Feel free to ask questions in the comments below.

A Beginner's Guide to Traefik Tuning for Rails Applications

With the Ingredient of Time, Any Small Bet Can Become a Big One Three years ago, I started , the acronym for , a website with information about movies available on Netflix ( in the US, UK, ES) with and rankings. Bonetflix Best on Netflix IMDB Filmaffinity This is how the website looked on December 2023 on Google Search Console in the last 12 months. I’m thrilled to share my journey on this project from inception to future. Start Everything originated from a confluence of compelling reasons. As a dedicated enthusiast of movies and series, I embarked on this venture prompted by the challenge of finding a ranking for the available Netflix titles in Spain. I researched keywords related to Netflix on Keyword Planner. I identified a substantial volume of searches incorporating the query "$movie_name Netflix." At that juncture, Netflix lacked the public movie pages that characterize its user experience. This is the first project I did targeting the web as the main platform; a departure from my usual terrain of Android Apps. Despite the existence of competitors in the field, I decided to embark on this project with a focused objective: to establish a robust online presence. Build In developing the website, I opted to utilize HTML, CSS, and minimal Javascript to ensure optimal speed and performance. Everything is static content following some of the principles of “ .” The current score for the website is as follows: Manifesto for Preserving Content on the Web Lighthouse In the data phase, information was sourced from various outlets such as Netflix, Filmaffinity, and IMDB. Addressing website modifications and circumventing captchas were integral aspects of this endeavor. I also had to match these data between those different data sources, converting text to a vector, clustering, and identifying the most proximate vectors. A challenging aspect was discerning movies with identical names in the same or different years, as well as identifying similar actors in distinct films. With the amassed data, I decided not to make a copy-paste about the text extracted from different sources. So, I used the pre-ChatGPT trick to generate data, translated English movie overviews to Spanish to get Spanish overviews, and vice versa. Subsequently, I also read and reviewed the translations, recognizing that they were not consistently accurate and might also include errors today. Marketing/SEO The first year was hard; I felt like Google didn’t trust my website and didn’t rank the keywords. However, as time progressed, there was a discernible shift, indicating a gradual gain in trust. I don’t have data from the start as Google Search Console doesn’t allow me to get data older than 16 months (with this newsletter, things won’t be lost). The top linking sites contributing to website visibility are the following: Furthermore, I built the to explore the potential impact on results through website referencing (I will not delve into the Android app, maybe in another issue). That’s why I have Google as a top linking site. Bonetflix App This initiative faced initial setbacks, as Google penalized the behavior in the early weeks, resulting in diminished site visits. Subsequently, however, normalcy was restored. Another experimentation involved referencing select movies from Wikipedia, even with the inclusion of a . While there was a transient surge in website ranking, this effect did not endure over the long term. "no follow" backlink As the website continues to mature and solidify its standing, I focused on effective monetization strategies. The following paragraphs are my experience. Monetization Seeking the viability of the project, I decided to add ads. I chose to add and . Google Adsense Auto ads Amazon Prime Video Affiliate Program Adsense Auto ads choose the best place on a webpage to place ads based on different elements such as the page layout and the amount of content on the page. Everything to increase your revenue, and Google’s revenue, for sure. However, the ad experience was not very pleasant and hurt the user experience. I ran the experiment in March, and these are the results: I was far away from sustainability with Auto ads, but Amazon Prime Video didn’t go better. I placed an Amazon Prime Video subscription banner near the “Watch on Netflix” button; here is example. But it resulted in 0 Prime Video subscriptions. The Godfather Netflix With these results and the harm to user experience. I decided to remove Google Ads and set up Prime Video ads seeking any conversion (€3 per subscribed user). But I saw that with this traffic, I wouldn’t be able to achieve sustainability as I’d need x500 more traffic. Costs I used , , and services from AWS  for website infrastructure. These are the costs of this last year: Route 53 S3 Cloudfront I’m gathering available movie and show data from Netflix. To gather this data, I need to be a Netflix user, and Netflix’s Standard plan is €12,99/month in Spain. A total of $197.01 a year. Future I devoted a substantial timeframe, specifically three years, to the development of this project, with two of those years being concurrently managed alongside my full-time employment. What began as a small bet evolved into a considerable bet in terms of time. My ambitions for the project extend to the integration of a movie search function, enhanced movie rankings, and the inclusion of trailers to augment the overall user experience. However, achieving these objectives necessitates a more dedicated allocation of time on my part. Nowadays, there are movie rankings platforms with their filter of movies available on Netflix, even Google shows its results in the first search position. Sometimes, I have thoughts of forgetting the project and dedicating my time to something else. While the work brings me joy, its financial returns have yet to reach a limit that can provide a sustainable livelihood, and I doubt they will be able to in the future. I have other projects in mind that would benefit from my time dedicated to Bonetflix. However, I find value in maintaining this project. It serves as a valuable instrument for the exploration of new concepts and ideas, building an audience, and enhancing search engine optimization (SEO). In essence, the project holds the potential to significantly contribute to my ongoing learning curve and overall skill development. Also published here

Taking a Big Bet on Yourself and Your Project

নিজেকে এবং আপনার প্রকল্পের উপর একটি বড় বাজি নেওয়া

Gehen Sie eine große Wette auf sich selbst und Ihr Projekt ein

Hacer una gran apuesta por usted mismo y su proyecto

Prendre un gros pari sur vous-même et votre projet

अपने आप पर और अपने प्रोजेक्ट पर बड़ा दांव लगाना

自分自身と自分のプロジェクトに大きな賭けをする

자신과 프로젝트에 큰 투자를 하세요

Fazendo uma grande aposta em você e no seu projeto

Делаем большую ставку на себя и свой проект

Kendinize ve Projenize Büyük Bahse Girmek

Đặt cược lớn vào bản thân và dự án của bạn

对自己和项目下大赌注

On the seventh day, amidst the darkness, God granted ProjectDiscovery, aiding brave souls in their perilous pen-testing journey. In the wake of ProjectDiscovery's ascent, tools like Nuclei, Katana, Subfinder, Naabu, and the latest addition, cvemap, emerged to fortify the arsenal of the brave. CVEmap and Nuclei missing feature And most of the were disappointed by the missing feature to “map” results directly to existing nuclei templates. penetration testers cvemap This absolutely necessary feature does not exist today, and I hope it will change - I will create an issue on to address this problem. GitHub However, I decided to fix this problem in this guide myself because a solution might not come at all. Fortunately, the existing nuclei templates use the same naming convention as the CVE one, and nuclei support an argument where you can enter the ID of the desired templates directly into the input.  CVEmap’s flag  nicely informs that the template exists in the Nuclei, but unfortunately, it doesn't tell how and where to find it. -template By combining this information, I came up with an ugly but short solution. Solution A bit of allows us to create the following script. bash glue This script scans targets from a file, with Nuclei templates filtered based on criteria from cvemap. Command expansion ( ) creates the resulting list of template IDs, separated by a comma, which Nuclei accepts as valid template selection. targets.txt $(…) nuclei -l "targets.txt" \
       -id "$(cvemap -template -list-id -silent \
        -age '<365' -q '"remote code execution" is_remote:true' -vendor cisco |
        tr -s '\n' ',')" \
       -o "output.txt" Precisely, in this case, the command scans CVEs of the Cisco products that provide RCE to the attackers and are less than one year old. Isn't that great!? Feel free to change the query on line 3 for your purpose. The other lines are mandatory. Postface Why this feature doesn't exist officially, I don't know, but if I had to bet, it's because they want to use the synergy of these tools in their paid cloud platform, and they don't want to give all the features for free but at the same time they want to be opensource. So they created a flag with the flag, which informs you that the template exists, but which one it is, and so on, you must look up yourself. cvemap -template Resources https://docs.projectdiscovery.io/tools/cvemap/running https://cloud.projectdiscovery.io/?ref=api_key https://github.com/projectdiscovery/cvemap https://github.com/projectdiscovery/nuclei

The Pen-Tester's Arsenal: ProjectDiscovery's CVEmap to Nuclei Template Mapping

How to Optimize Your Website For Higher Search Engine Rankings

The New York Times Company v. Microsoft Corporation Court Filing December 27, 2023 is part of . You can jump to any part in this filing . This is part 10 of 27. HackerNoon’s Legal PDF Series here IV. FACTUAL ALLEGATIONS C. Defendants’ Unauthorized Use and Copying of Times Content 82. Microsoft and OpenAI created and distributed reproductions of The Times’s content in several, independent ways in the course of training their LLMs and operating the products that incorporate them. 1. Unauthorized Reproduction of Times Works During GPT Model Training 83. Defendants’ GPT models are a family of LLMs, the first of which was introduced in 2018, followed by GPT-2 in 2019, GPT-3 in 2020, GPT-3.5 in 2022, and GPT-4 in 2023. The “chat” style LLMs, GPT-3.5 and GPT-4, were developed in two stages. First, a transformer model was pre-trained on a very large amount of data. Second, the model was “fine-tuned” on a much smaller supervised dataset in order to help the model solve specific tasks. 84. The pre-training step involved collecting and storing text content to create training datasets and processing that content through the GPT models. While OpenAI did not release the trained versions of GPT-2 onward, “[d]ue to [OpenAI’s] concerns about malicious applications of the technology,” OpenAI has published general information about its pre-training process for the GPT models.[12] 85. GPT-2 includes 1.5 billion parameters, which was a 10X scale up of GPT.[13] The training dataset for GPT-2 includes an internal corpus OpenAI built called “WebText,” which includes “the text contents of 45 million links posted by users of the ‘Reddit’ social network.”[14] The contents of the WebText dataset were created as a “new web scrape which emphasizes document quality.”[15] The WebText dataset contains a staggering amount of scraped content from The Times. For example, the NYTimes.com domain is one of the “top 15 domains by volume” in the WebText dataset,[16] and is listed as the 5th “top domain” in the WebText dataset with 333,160 entries.[17] 86. GPT-3 includes 175 billion parameters and was trained on the datasets listed in the table below.[18] 87. One of these datasets, WebText2, was created to prioritize high value content. Like the original WebText, it is composed of popular outbound links from Reddit. As shown in the table above, the WebText2 corpus was weighted 22% in the training mix for GPT-3 despite constituting less than 4% of the total tokens in the training mix. Times content—a total of 209,707 unique URLs—accounts for 1.23% of all sources listed in OpenWebText2, an open-source re-creation of the WebText2 dataset used in training GPT-3. Like the original WebText, OpenAI describes WebText2 as a “high-quality” dataset that is “an expanded version of the WebText dataset … collected by scraping links over a longer period of time.”[19] 88. The most highly weighted dataset in GPT-3, Common Crawl, is a “copy of the Internet” made available by an eponymous 501(c)(3) organization run by wealthy venture capital investors.[20] The domain www.nytimes.com is the most highly represented proprietary source (and the third overall behind only Wikipedia and a database of U.S. patent documents) represented in a filtered English-language subset of a 2019 snapshot of Common Crawl, accounting for 100 million tokens (basic units of text): [21] 89. The Common Crawl dataset includes at least 16 million unique records of content from The Times across News, Cooking, Wirecutter, and The Athletic, and more than 66 million total records of content from the Times. 90. Critically, OpenAI admits that “datasets we view as higher-quality are sampled more frequently” during training.[22] Accordingly, by OpenAI’s own admission, high-quality content, including content from The The Times, was more important and valuable for training the GPT models as compared to content taken from other, lower-quality sources. 91. While OpenAI has not released much information about GPT-4, experts suspect that GPT-4 includes 1.8 trillion parameters, which is over 10X larger than GPT-3, and was trained on approximately 13 trillion tokens.[23] The training set for GPT-3, GPT-3.5, and GPT-4 was comprised of 45 terabytes of data—the equivalent of a Microsoft Word document that is over 3.7 billion pages long. [24] Between the Common Crawl, WebText, and WebText2 datasets, the Defendants likely used millions of Times-owned works in full in order to train the GPT models. 92. Defendants repeatedly copied this mass of Times copyrighted content, without any license or other compensation to The Times. As part of training the GPT models, Microsoft and OpenAI collaborated to develop a complex, bespoke supercomputing system to house and reproduce copies of the training dataset, including copies of The Times-owned content. Millions of Times Works were copied and ingested—multiple times—for the purpose of “training” Defendants’ GPT models. 93. Upon information and belief, Microsoft and OpenAI acted jointly in the large-scale copying of The Times’s material involved in generating the GPT models programmed to accurately mimic The Times’s content and writers. Microsoft and OpenAI collaborated in designing the GPT models, selecting the training datasets, and supervising the training process. As Mr. Nadella stated: So, there are a lot of, I call it, product design choices one gets tomake when you think about AI and AI safety. Then, let’s come at itthe other way. You have to take real care of the pretrained data because models are trained on pretrained data. What’s the quality,the provenance of that pretrained data? That’s a place where we’vedone a lot of work.[25] 94. To the extent that Microsoft did not select the works used to train the GPT models, it acted in self-described “partnership” with OpenAI respecting that selection, knew or was willfully blind to the identity of the selected works by virtue of its knowledge of the nature and identity of the training corpuses and selection criteria employed by OpenAI, and/or had the right and ability to prevent OpenAI from using any particular work for training by virtue of its physical control of the supercomputer it developed for that purpose and its legal and financial influence over the OpenAI Defendants. 95. Upon information and belief, Microsoft and OpenAI continue to create unauthorized copies of Times Works in the form of synthetic search results returned by their Bing Chat and Browse with Bing products. Microsoft actively gathers copies of the Times Works used to generate such results in the process of crawling the web to create the index for its Bing search engine. 96. On information and belief, Microsoft and OpenAI are currently or will imminently commence making additional copies of Times Works to train and/or fine-tune the next-generation GPT-5 LLM. 97. Defendants’ large-scale commercial exploitation of Times content is not licensed, nor have Defendants received permission from The Times to copy and use its works to build their GenAI tools. Continue Reading . Here [12] OpenAI, Better Language Models and Their Implications, OPENAI (Feb. 14, 2019), https://openai.com/research/better-language-models. [13] Id. [14] GPT-2 Model Card, GITHUB (Nov. 2019), https://github.com/openai/gpt-2/blob/master/model_card.md. [15] RADFORD ET AL., LANGUAGE MODELS ARE UNSUPERVISED MULTITASK LEARNERS 3 (2018), https://d4mucfpksywv.cloudfront.net/better-language-models/language-models.pdf. [16] GPT-2 Model Card, supra note 14. [17] GPT-2 / domains.txt, GITHUB, https://github.com/openai/gpt-2/blob/master/domains.txt (last visited Dec. 21, 2023). [18] BROWN ET AL., LANGUAGE MODELS ARE FEW-SHOT LEARNERS 9 (2020), https://arxiv.org/pdf/2005.14165.pdf. [19] Id. at 8. [20] COMMON CRAWL, https://commoncrawl.org/ (last visited Dec. 21, 2023). [21] DODGE ET AL., DOCUMENTING LARGE WEBTEXT CORPORA: A CASE STUDY ON THE COLOSSAL CLEAN CRAWLED CORPUS (2021), https://arxiv.org/abs/2104.08758. [22] BROWN ET AL., supra note 18. [23] Maximilian Schreiner, GPT-4 Architecture, Datasets, Costs and More Leaked, THE DECODER (July 11, 2023), https://the-decoder.com/gpt-4-architecture-datasets-costs-and-more-leaked/. [24] Kindra Cooper, OpenAI GPT-3: Everything You Need to Know [Updated], SPRINGBOARD (Sept. 27, 2023), https://www.springboard.com/blog/data-science/machine-learning-gpt-3-open-ai/. [25] Nilay Patel, Microsoft Thinks AI Can Beat Google at Search — CEO Satya Nadella Explains Why, THE VERGE (Feb. 7, 2023), https://www.theverge.com/23589994/microsoft-ceo-satya-nadella-bing-chatgpt-googlesearch-ai. About HackerNoon Legal PDF Series: We bring you the most important technical and insightful public domain court case filings. This court case 1:23-cv-11195 retrieved on December 29, 2023, from is part of the public domain. The court-created documents are works of the federal government, and under copyright law, are automatically placed in the public domain and may be shared without legal restriction. nycto-assets.nytimes.com

Stories, News & Mentions about CLOUDNURO

Cloudnuro’s Stories on HackerNoon