GET DUCK

getduck

174 AI Tools To Try

When building projects one of the annoying parts is setting up stuff like web servers, relational databases, caching, etc. It’s often expensive (Heroku charges $25/month for their 1GB instance, the same server is 5/month on DigitalOcean) or tedious to set up and manage. I no longer enjoy spending hours and days setting up a server, building tools, sending code to the server, building it, getting an SSL certificate, installing it, update nginx over and over again whenever I have a new project. That’s where open-source PAASs came in. They often have an extremely easy-to-use app/database deployment & web server manager. Examples: , , etc. CapRover Dokku For example, with CapRover, you can host the below with one click: Databases and Database GUIs. Eg PostgreSQL, Redis, MySQL, MongoDB, etc. Email hosting, newsletter & mailing list solutions. Customer support, CMSs, ERPs, CRMs, LMSs & Invoicing solutions. Analytics Blogging and Content. Eg: Ghost, Jekyll, WordPress, etc. CI/CD Dev tools (monitoring, notifications, URL shorteners, backups, etc) Cloud storage, FTP & media servers Torrent clients Document servers (text documents, spreadsheets & presentations) And more wrapped in a Docker container Literally anything Requirements: Own a domain. Have some familiarity with the Cloud, Linux and Docker. Get server & install Docker Set up an Ubuntu VPS Good providers: (Free ) https://cloud.google.com/free/docs/gcp-free-tier/#compute (Free if you can to sign up lol) https://www.oracle.com/ke/cloud/free/#always-free manage https://contabo.com/en/vps/ https://www.netcup.eu/vserver/vps.php#v-server-details https://www.ovhcloud.com/en/vps/compare/ https://www.racknerd.com/kvm-vps https://www.hetzner.com/cloud Create VPS Instance I’ll be using the free virtual machine ( ) from Google Cloud Platform. Feel free to use any other. Compute Engine In GCP Create an Ubuntu instance from the . GCP Dashboard Configure Firewall Add firewall rules to allow network traffic from the following: 80 TCP for regular HTTP connections 443 TCP for secure HTTPS connections 3000 TCP for initial Installation (can be blocked once attached to a domain) 7946 TCP/UDP for Container Network Discovery 4789 TCP/UDP for Container Overlay Network 2377 TCP/UDP for Docker swarm API 996 TCP for secure HTTPS connections specific to Docker Registry In case of an ubuntu server, run ufw allow 80,443,3000,996,7946,4789,2377/tcp; ufw allow 7946,4789,2377/udp; Your VPS provider may have a different way to configure the firewall. In GCP In the case of GCP, we can create firewall rules from the : Networking Dashboard Note that I’ve created a network tag that we’ll use next. caprover From the , edit the virtual machine we created, adding the network tag. Compute Engine dashboard caprover Install Docker SSH into the VM and install Docker. This can be done using the get : convenience script curl -fsSL https://get.docker.com -o get-docker.sh

sudo sh get-docker.sh Sample output: Set up CapRover Install CapRover on the server Run the following to install CapRover: sudo docker run -p 80:80 -p 443:443 -p 3000:3000 -v /var/run/docker.sock:/var/run/docker.sock -v /captain:/captain caprover/caprover NOTE: do not change the port mappings. CapRover only works on the specified ports. You will see a bunch of outputs on your screen. Once the CapRover is initialised, you can visit in your browser and log in to CapRover using the default password . You can change your password later. However, do not make any changes in the dashboard. We'll use the command-line tool to set up the server. http://[IP_OF_YOUR_SERVER]:3000 captain42 Set up DNS Let's say you own . Set as an in your DNS settings to point to the IP address of the server where you installed CapRover. mydomain.com *.something.mydomain.com A-record Find more info . here Install CapRover CLI on your local machine npm install -g caprover Complete CapRover Setup & log in to the Dashboard Complete CapRover Setup Run the following on your local machine: caprover serversetup Follow the steps and log in to your CapRover instance. When prompted to enter the root domain, enter assuming that you set to point to your IP address in step #Set up DNS. Now you can access your CapRover from and log in. something.mydomain.com *.something.mydomain.com captain.something.mydomain.com Log in to the Dashboard Visit the CapRover dashboard at and log in using the password you set up in the above step. http://captain.something.mydomain.com Install the One-Click Apps from the Dashboard Sources and More Information https://caprover.com/docs/get-started.html#step-1-caprover-installation https://cloud.google.com/compute/docs

Set Up Your Own Heroku-like Cloud (PAAS) in 10 Minutes: A Quick Start Guide

Configure su propia nube similar a Heroku (PAAS) en 10 minutos: una guía de inicio rápido

10 मिनट में अपना खुद का हरोकू जैसा क्लाउड (PAAS) सेट करें: एक त्वरित शुरुआत गाइड

This is a pretty straightforward blog and personal documentation on how to set up an SFTP server with s3 bucket as the backend of the SFTP server. I hope you find this helpful as I did. Step 1. Create S3 Bucket from the AWS console. The S3 Bucket will be the place where the files will stay. No custom changes will need to be made to the s3 bucket itself. Step 2. Navigate to AWS Transfer Family service and select .  Where you will be prompted to select , , .  Select SFTP, which stands for SSH File Transfer Protocol. SFTP is a network protocol that provides file access, file transfer, and file management over any reliable data stream. create Server FTP FTPS SFTP Step 3. Next, we need to select an identity provider, which in short is an access layer that will allow certain users to access the SFTP server. You can do this programmatically via lambda and add your own custom identity provider, or in this case, just create a single user at a time use the AWS managed service Step 4. Next, we select and set up our own custom DNS hostname. In this case, we are just going to use AWS generated host that is provided. If you would like this could be modified in the future. Options: None: AWS will create a DNS record such as {some-random-generated-string}.server.transfer.{region}.amazonaws.com. Route 53 Alias: the service will create an alias on Route 53 so you can use a more - friendly name and a subdomain such as . ftps.martinpatino.com Other DNS: 3rd party DNS server option. Step. 5 Just select a domain, which will provide 2 options, here we will just go with Amazon S3 as we will provide user access to an s3 bucket with limited permissions to a specific bucket. Once this step is completed, you will be prompted to review your server summary and confirm, in which your SFTP server will be generated. Step. 6 Configuration of IAM Roles with S3 So after we have created the SFTP server and created the S3 bucket that you would like the user to have access. The next part is to handle user role permissions and policy creation. In our case, we want to restrict users to only being able to view a specific bucket. So just head to IAM create a custom SFTP role for your user in AWS under the service use case of . Transfer You can copy and paste this and modify the field. custom-bucket-name This rule will give the user access to delete, fetch, update and add files to your s3 bucket via FTP. {
    "Version": "2012-10-17",
    "Statement": [
        {
            "Action": [
                "s3:ListBucket",
                "s3:GetBucketLocation"
            ],
            "Resource": [
                "arn:aws:s3:::{custom-bucket-name}"
            ],
            "Effect": "Allow",
            "Sid": "ReadWriteS3"
        },
        {
            "Action": [
                "s3:PutObject",
                "s3:GetObject",
                "s3:DeleteObject",
                "s3:DeleteObjectVersion",
                "s3:GetObjectVersion",
                "s3:GetObjectACL",
                "s3:PutObjectACL"
            ],
            "Resource": [
                "arn:aws:s3:::{custom-bucket-name}/*"
            ],
            "Effect": "Allow",
            "Sid": ""
        }
    ] Step.  7 To create a user you will need a username, s3 bucket you want the user to have access to, and the role they are associated with, which would be the one that was created above. Once you bind your new SFTP_COMPANY_ROLE or whatever you called it to the user. In my example below, I am just calling user , but you can call it whatever you want. You will then have the option to set up a policy, which is not required. However, it will help lock down your user to only be able to have access to a specific directory under a certain bucket. usercompany_a AWS Transfer does have an auto-generate policy if you would like to use it, which would look like the example below. {
  "Version": "2012-10-17",
  "Statement": [
    {
      "Sid": "AllowListingOfUserFolder",
      "Action": [
        "s3:ListBucket"
      ],
      "Effect": "Allow",
      "Resource": [
        "arn:aws:s3:::${transfer:HomeBucket}"
      ],
      "Condition": {
        "StringLike": {
          "s3:prefix": [
            "${transfer:HomeFolder}/*",
            "${transfer:HomeFolder}"
          ]
        }
      }
    },
    {
      "Sid": "HomeDirObjectAccess",
      "Effect": "Allow",
      "Action": [
        "s3:PutObject",
        "s3:GetObject",
        "s3:DeleteObject",
        "s3:GetObjectVersion"
      ],
      "Resource": "arn:aws:s3:::${transfer:HomeDirectory}*"
    }
  ]
} The final step is add SSH Public Key, which the user should provide to you. The user will the use their own private key to connect to the SFTP via some SFTP client. This is a required field as without it. Your users will not  be able to connect. The last step is to add the user’s Public Key. They will need their Private Key to connect to the SFTP Server. If you don’t add the Public Key, the users can connect without any credentials. Test Time It’s time to test the SFTP connection. I use CyberDuck, which is available on both Mac/Windows. Try to connect and if everything is setup correctly, you should be able to see the content in you S3 bucket. You should even be able to upload files from your SFTP client. Success!!! Hope you enjoy learning a bit! First Published here

How to Set Up an SFTP Server with S3 

Photo by on dasha shchukova Unsplash Intro What is Buck? Buck is a free, open-source and lightweight tool used by developers to group multiple terminal commands into one. CLI Starting a new project, installing packages, executing, updating, and deploying requires you to run multiple terminal commands (for example:) Depending on your use case, this can range from a few commands to several commands which can be overwhelming and it distract us from actually building the next big thing. Especially if you are like me and you just want to show your parents that you are actually doing something with your life. Running these commands over and over again for your new projects is pretty inefficient and it goes against a rule we elite programmers swear by: Do not repeat yourself (DRY). Buck solves this problem by allowing you to group multiple commands once into buckets and you can run them with a custom keyword of your choice. Proudly made in 🇳🇬 by  Pleasanttech💚 Getting Started https://docs.getbuck.tech/ Also published here.

How to Run Multiple Terminal Commands in ONE, Like a Boss

Automated Data Replication From AWS S3 To Microsoft Azure Storage Made Easy

I am sure that you already have Zoom, Slack and Asana on your computer. But daily office life is more than written tasks and planned meetings. There are lunches with your team, spontaneous talks near the water cooler, common tasks and goals you want to acheive. It’s your comfortable workplace and it by itself motivates you to be productive. Just look at the bunch of stickers on the wall — big things are waiting for you! After we all switched to remote work, many of these processes disappeared from our lives. But it seems to me that we can move them online . Life in office has a lot of invisible and important processes in it. painlessly So let’s talk about these processes and the tools that help us recreate the office vibe at home! The ones that help remote teams keep in touch, talk to each other more and don’t miss the opportunities they could have in the office. p.s. You will not find tools here. And I did it in purpose. very popular How to create the atmosphere of the office home Try TandemChat . it’s a virtual office — you can quickly interact with colleagues, spontaneously call them and work together on the same platforms (Trello, Notion, Google Docs, etc.) Description: , , , [not launched yet], [not launched yet], [not launched yet]. Analogs: RemoteHQ Teemly Pragli Makespace With wfhland How to recreate the feeling of the meeting in one place Try HighFidelity . virtual spaces that simulate real meetings. Description: , . Analogs: Connect.club Airmeet How to recreate the small talks before the meetings Try Brightful . short games / quizzes that are designed to help the participants of the meeting to find connection with each other in a friendly way. Description: How to replace the meeting-boards that help you find the best decisions Try Ment . A visual platform that simplifies decision-making for teams. Description: How to recreate the opportunity to meet colleagues spontaneously (and not only colleagues) Try . NextPage An app that allows you to meet spontaneously online with colleagues and friends. Description: How to recreate the tradition to put stickers on the board Try Parabol . : software for conducting online retrospectives. Description How to transfer the coffee breaks from offline to online With Hallway . A Slack bot that automatically allocates time for joint coffee breaks. Description: How to recreate the code cross-editing process Try GitDuck . : tools for developers that enables to talk, share your code and do pair programming with ease. Description How to recreate the feeling of being idle in front of colleagues Try Focusmate . “virtual coworking” — where you can easily find a “colleague”, and monitor the concentration and productivity of each other. Description: How to replace meeting moderator Try Macro . : an add-on for video calls services (Zoom, Meet) that helps make meetings more efficient. Description How to recreate the daily stand-up’s with your teammates Try Standups . “stories for stand-ups” — update each other regardless of their time zone or location, using async video & voice messages. Description: How to replace assistant who makes meeting notes Try Hugo . : An app that makes it easy to create, store, and collaborate on notes after a meeting. Description How to recreate an opportunity to share emotions during meetings Try . Nod extension for Google Chrome that allows you to exchange emotions during Google Meet calls. Description: How to replace assistant who helps you with your schedule Try . Clockwise : An app for G-Suite that optimizes your calendar, searching for a best time for a focused work. Description How to replace meetings, yep, all the meetings (even virtual ones) Try [not launched yet, but seems promising — from creators of Meerkat и Houseparty] SlashTalk : it is not clear yet, but their manifesto says that they want to decentralize communication within the teams completely. Description

20+ Extraordinary Apps For Remote Teams

In this article, I’m will talk about how you can build a application using (SAM) to perform Log Analytics on AWS data using . The Serverless Application will help you analyze CloudTrail Logs using Amazon Elasticsearch Service. The application creates CloudTrail trail, sets the log delivery to an s3 bucket that it creates and configures SNS delivery whenever the CloudTrail log file has been written to s3. The app alsocreates an Amazon Elasticsearch Domain and creates an Amazon Lambda Function which gets triggered by the SNS message, get the s3 file location, read the contents from the s3 file and write the data to Elasticsearch for analytics. Serverless AWS Serverless Application Model CloudTrail Amazon Elasticsearch Service AWS AWS Let’s learn about what is AWS CloudTrail, Elasticsearch, Amazon Elasticsearch Service, AWS Lambda and AWS SAM. What is AWS CloudTrail? AWS CloudTrail is a service that enables governance, compliance, operational auditing, and risk auditing of your AWS account. With CloudTrail, you can log, continuously monitor, and retain account activity related to actions across your AWS infrastructure. CloudTrail provides event history of your AWS account activity, including actions taken through the AWS Management Console, AWS SDKs, command line , and other AWS services. This event history simplifies security analysis, resource change tracking, and troubleshooting. tools _AWS CloudTrail allows you track and automatically respond to account activity threatening the security of your AWS…_aws.amazon.com AWS CloudTrail What is Elasticsearch? Elasticsearch is a distributed, RESTful search and analytics engine capable of solving a growing number of use cases. As the heart of the Elastic Stack, it centrally stores your data so you can discover the expected and uncover the unexpected. _Distributed, open source search and analytics engine designed for horizontal scalability, reliability, and easy…_www.elastic.co Elasticsearch: RESTful, Distributed Search & Analytics | Elastic What is Amazon Elasticsearch Service? Amazon Elasticsearch Service makes it easy to deploy, secure, operate, and scale Elasticsearch for log analytics, full text search, application monitoring, and more. Amazon Elasticsearch Service is a fully managed service that delivers Elasticsearch’s easy-to-use APIs and real-time analytics capabilities alongside the availability, scalability, and security that production workloads require. _Use Amazon Elasticsearch Service to easily deploy, operate and scale Elasticsearch on AWS. Launch your Amazon…_aws.amazon.com Amazon Elasticsearch Service — Amazon Web Services (AWS) What is AWS Lambda? AWS Lambda lets you run code without provisioning or managing servers. You pay only for the compute time you consume — there is no charge when your code is not running. With Lambda, you can run code for virtually any type of application or backend service — all with zero administration. Just upload your code and Lambda takes care of everything required to run and scale your code with high availability. You can set up your code to automatically trigger from other AWS services or call it directly from any web or mobile app. _AWS Lambda lets you run code without provisioning or managing servers. You pay only for the compute time you consume._aws.amazon.com AWS Lambda — Serverless Compute — Amazon Web Services What is AWS Serverless Application Model? AWS Serverless Application Model (AWS SAM) prescribes rules for expressing Serverless applications on AWS. The goal of AWS SAM is to define a standard application model for Serverless applications. _serverless-application-model — AWS Serverless Application Model (AWS SAM) prescribes rules for expressing Serverless…_github.com awslabs/serverless-application-model Now let’s look at how we can build a Serverless App to perform Log Analytics on AWS CloudTrail data using Amazon Elasticsearch Service. This is the architecture of the CloudTrail Log Analytics Serverless Application: Architecture for Serverless Application: CloudTrail Log Analytics using Elasticsearch AWS Serverless Application Model is a AWS template. Before we look at the code for SAM template, let’s work on packaging our AWS Lambda. Cloudformation On your workstation, create a working folder for building the Serverless Application. Create a file called index.py for the AWS Lambda: """ This module reads the SNS message to get the S3 file location for cloudtraillog and stores into Elasticsearch. """ from __future__ import print_functionimport jsonimport boto3import loggingimport datetimeimport gzipimport urllibimport osimport traceback from StringIO import StringIOfrom exceptions import * # from awses.connection import AWSConnectionfrom elasticsearch import Elasticsearch, RequestsHttpConnectionfrom requests_aws4auth import AWS4Auth logger = logging.getLogger()logger.setLevel(logging.INFO) s3 = boto3.client('s3', region_name=os.environ['AWS_REGION']) awsauth = AWS4Auth(os.environ['AWS_ACCESS_KEY_ID'], os.environ['AWS_SECRET_ACCESS_KEY'], os.environ['AWS_REGION'], 'es', session_token=os.environ['AWS_SESSION_TOKEN'])es = Elasticsearch(hosts=[{'host': os.environ['es_host'], 'port': 443}],http_auth=awsauth,use_ssl=True,verify_certs=True,connection_class=RequestsHttpConnection) def handler(event, context):logger.info('Event: ' + json.dumps(event, indent=2)) s3Bucket = json.loads(event\['Records'\]\[0\]\['Sns'\]\['Message'\])\['s3Bucket'\].encode('utf8')  
s3ObjectKey = urllib.unquote\_plus(json.loads(event\['Records'\]\[0\]\['Sns'\]\['Message'\])\['s3ObjectKey'\]\[0\].encode('utf8'))

logger.info('S3 Bucket: ' + s3Bucket)  
logger.info('S3 Object Key: ' + s3ObjectKey)

try:  
    response = s3.get\_object(Bucket=s3Bucket, Key=s3ObjectKey)  
    content = gzip.GzipFile(fileobj=StringIO(response\['Body'\].read())).read()

    for record in json.loads(content)\['Records'\]:  
        recordJson = json.dumps(record)  
        logger.info(recordJson)  
        indexName = 'ct-' + datetime.datetime.now().strftime("%Y-%m-%d")  
        res = es.index(index=indexName, doc\_type='record', id=record\['eventID'\], body=recordJson)  
        logger.info(res)  
    return True  
except Exception as e:  
    logger.error('Something went wrong: ' + str(e))  
    traceback.print\_exc()  
    return False Create a file called requirements for the python packages that are needed: elasticsearch>=5.0.0,<6.0.0requests-aws4auth With the above requirements file created in your workspace, run the below command to install the required packages: python -m pip install -r requirements.txt -t ./ Create a file called template.yaml that will store the code for AWS SAM: AWSTemplateFormatVersion: '2010-09-09'Transform: 'AWS::Serverless-2016-10-31'Description: > This SAM example creates the following resources:

  S3 Bucket: S3 Bucket to hold the CloudTrail Logs  
  CloudTrail: Create CloudTrail trail for all regions and configures it to delivery logs to the above S3 Bucket  
  SNS Topic: Configure SNS topic to receive notifications when the CloudTrail log file is created in s3  
  Elasticsearch Domain: Create Elasticsearch Domain to hold the CloudTrail logs for advanced analytics  
  IAM Role: Create IAM Role for Lambda Execution and assigns Read Only S3 permission  
  Lambda Function:  Create Function which get's triggered when SNS receives notification, reads the contents from s3 and stores them in Elasticsearch Domain Outputs: S3Bucket:  
  Description: "S3 Bucket Name where CloudTrail Logs are delivered"  
  Value: !Ref S3Bucket  
LambdaFunction:  
  Description: "Lambda Function that reads CloudTrail logs and stores them into Elasticsearch Domain"  
  Value: !GetAtt Function.Arn  
ElasticsearchUrl:  
  Description: "Elasticsearch Domain Endpoint that you can use to access the CloudTrail logs and analyze them"  
  Value: !GetAtt ElasticsearchDomain.DomainEndpoint Resources:SNSTopic:Type: AWS::SNS::TopicSNSTopicPolicy:Type: "AWS::SNS::TopicPolicy"Properties:Topics:- Ref: "SNSTopic"PolicyDocument:Version: "2008-10-17"Statement:-Sid: "AWSCloudTrailSNSPolicy"Effect: "Allow"Principal:Service: "cloudtrail.amazonaws.com"Resource: "*"Action: "SNS:Publish"S3Bucket:Type: AWS::S3::BucketS3BucketPolicy:Type: "AWS::S3::BucketPolicy"Properties:Bucket:Ref: S3BucketPolicyDocument:Version: "2012-10-17"Statement:-Sid: "AWSCloudTrailAclCheck"Effect: "Allow"Principal:Service: "cloudtrail.amazonaws.com"Action: "s3:GetBucketAcl"Resource:!Sub |-arn:aws:s3:::${S3Bucket}-Sid: "AWSCloudTrailWrite"Effect: "Allow"Principal:Service: "cloudtrail.amazonaws.com"Action: "s3:PutObject"Resource:!Sub |-arn:aws:s3:::${S3Bucket}/AWSLogs/${AWS::AccountId}/*Condition:StringEquals:s3:x-amz-acl: "bucket-owner-full-control"CloudTrail:Type: AWS::CloudTrail::TrailDependsOn:- SNSTopicPolicy- S3BucketPolicyProperties:S3BucketName:Ref: S3BucketSnsTopicName:Fn::GetAtt:- SNSTopic- TopicNameIsLogging: trueEnableLogFileValidation: trueIncludeGlobalServiceEvents: trueIsMultiRegionTrail: trueFunctionIAMRole:Type: "AWS::IAM::Role"Properties:Path: "/"ManagedPolicyArns:- "arn:aws:iam::aws:policy/service-role/AWSLambdaBasicExecutionRole"- "arn:aws:iam::aws:policy/AmazonS3ReadOnlyAccess"AssumeRolePolicyDocument:Version: "2012-10-17"Statement:-Sid: "AllowLambdaServiceToAssumeRole"Effect: "Allow"Action:- "sts:AssumeRole"Principal:Service:- "lambda.amazonaws.com"ElasticsearchDomain:Type: AWS::Elasticsearch::DomainDependsOn:- FunctionIAMRoleProperties:DomainName: "cloudtrail-log-analytics"ElasticsearchClusterConfig:InstanceCount: "2"EBSOptions:EBSEnabled: trueIops: 0VolumeSize: 20VolumeType: "gp2"AccessPolicies:Version: "2012-10-17"Statement:-Sid: "AllowFunctionIAMRoleESHTTPFullAccess"Effect: "Allow"Principal:AWS: !GetAtt FunctionIAMRole.ArnAction: "es:ESHttp*"Resource:!Sub |-arn:aws🇪🇸${AWS::Region}:${AWS::AccountId}:domain/cloudtrail-log-analytics/*-Sid: "AllowFullAccesstoKibanaForEveryone"Effect: "Allow"Principal:AWS: "*"Action: "es:*"Resource:!Sub |-arn:aws🇪🇸${AWS::Region}:${AWS::AccountId}:domain/cloudtrail-log-analytics/_plugin/kibanaElasticsearchVersion: "5.5"Function:Type: 'AWS::Serverless::Function'DependsOn:- ElasticsearchDomain- FunctionIAMRoleProperties:Handler: index.handlerRuntime: python2.7CodeUri: ./Role: !GetAtt FunctionIAMRole.ArnEvents:SNSEvent:Type: SNSProperties:Topic: !Ref SNSTopicEnvironment:Variables:es_host:Fn::GetAtt:- ElasticsearchDomain- DomainEndpoint Packing Artifacts and uploading them to s3: Run the following command to upload your artifacts to S3 and output a packaged template that can be readily deployed to CloudFormation. aws cloudformation package \--template-file template.yaml \--s3-bucket bucket-name \--output-template-file serverless-output.yaml Deploying this AWS SAM to AWS CloudFormation: You can use CLI command to deploy the SAM template. Under-the-hood it creates and executes a changeset and waits until the deployment completes. It also prints debugging hints when the deployment fails. Run the following command to deploy the packaged template to a stack called : aws cloudformation deploy cloudtrail-log-analytics aws cloudformation deploy \--template-file serverless-output.yaml \--stack-name cloudtrail-log-analytics \--capabilities CAPABILITY_IAM Refer to the for more details. documentation I recommend reading about Elasticsearch Service Access Policies using the and modify the Access policy of the Elasticsearch domain to further fine tune the access policy. documentation Once the Serverless application is deployed in your AWS account, It will automatically store the AWS CloudTrail data into Amazon Elasticsearch Service as soon as the log is delivered to s3. With the data in Elasticsearch, you can use Kibana to visualize the data in Elasticsearch and create the dashboards that you need on the AWS CloudTrail data. The above Serverless Application Model app is available at the below Github repo: _cloudtrail-log-analytics — Cloudtrail Log Analytics using Amazon Elasticsearch Service — AWS Serverless Application_github.com ExpediaDotCom/cloudtrail-log-analytics

Serverless App: AWS CloudTrail Log Analytics using Amazon Elasticsearch Service

Laravel & docker This article is a step-by-step to get started with Vessel ( the new way to setup the Laravel setup config with docker). If your are not familiar with docker you can check my last article “ ”. A step by step guide to setup PHP (Laravel) environment (Linux) It is a long time that I am a big fan of chris fideo the author of ,one of the most rich source of content to learn server’s configurations , security ,database,docker and much more DevOps topics. servers for hackers is one of the amazing free course, it’s a step-by-step docker workflow to develop a breeze for your application. According to Chris,Vessel is the result of this docker workflow more is an official that everyone could easily use. Dockerizing your Application Let’s give it a try Vessel is a lightweight docker environment for Laravel, that means it is just a small set of files that set up a local docker development environment per project and there is no extra thing to install globally, expect Docker system itself. Install Docker and Docker-compose Docker is an open source project based on Linux container, if you are not familiar yet with docker, this article “ ” by is the ideal place to take your first step. A Beginner-Friendly Introduction to Containers, VMs and Docker Preethi Kasireddy At this moment Vessel only works on Mac and Linux because it uses a bash script to run Docker commands. For Linux Ubuntu distribution the easiest way to install docker is by the following: curl -fsSL get.docker.com | sudo shsudo usermod -aG docker your-user // add your user to docker group To Install docker compose run this command to download the latest version : sudo curl -L -s`-`uname -m` -o /usr/local/bin/docker-compose https://github.com/docker/compose/releases/download/1.17.0/docker-compose-`uname Next,Apply executable permissions to the binary. sudo chmod +x /usr/local/bin/docker-compose Learn more about docker installation : , docker docker-compose Get Started with Vessel First you need to create a Laravel project to start with by the following : composer create-project — prefer-dist laravel/laravel vessel_test Now and as I mention before Vessel is lightweight docker environment includes a Laravel service provider that makes easy and simple to start your own development setup. All you need to do in order to start your dev setup with vessel : composer require shipping-docker/vessel # Install Vessel into your project php artisan vendor:publish — provider=”Vessel\VesselServiceProvider” # Publish the `vessel` command and Docker files bash vessel init # Run this once to initialize project # Must run with “bash” until initialized The first time you start up an application with Vessel using the init command, vessel will download the following official Docker images: PHP 7.1 MySQL 5.7 Redis NodeJS with NPM, Yarn, and Gulp Start vessel by running : cd vessel_test./vessel start Head to in your browser and see your Laravel site! http://localhost And at last, also Vessel has good documentation that covers everything you need to know to get started : official documentation Bonus : Vessel Cheat Sheet Thanks for reading! If you think other people should read this, clap for me, tweet and share the post. Remember to follow me on Medium so you can get notified about my future posts. Read more stories https://elazizi.com/ P.P.S. if you liked this,give me at least dozen claps for good luck 👌👌.

Laravel & Docker, Zero config with Vessel

In this article I will guide you through a full installation and setup of application. Servicebot is an open-source platform that will sell any service online. Servicebot Here is our Github repository. Prerequisites Make sure to have the following items before starting the installation: I used the smallest Ubuntu 16.04 droplet in DigitalOcean for this installation. You could also just use a local server to test this out. Linux server. It is free and you need it to be able to process payments with your application. Stripe account. Servicebot Servicebot will use this for sending out automated emails to your clients and yourself. (Optional) Mail Server. . Stripe requires SSL certificate for live mode payments. One way you can obtain a SSL cert is using . Make sure to name your certificate as mentioned below. (Optional) SSL Cert Let’s Encrypt Certificates should be named as follows:- Keyfile : servicebot.key- Certificate : servicebot.crt- CA : servicebot_bundle.crt Once you have the above prerequisites, move on to the installation. Install Servicebot SSH into your server and run the following commands to install your Servicebot application. 1- Install docker if not installed already sudo wget -qO- https://get.docker.com/ | sh 2- Install docker-compose sudo curl -o /usr/local/bin/docker-compose -L "https://github.com/docker/compose/releases/download/1.11.2/docker-compose-$(uname -s)-$(uname -m)" 3- Set the permissions for docker-compose sudo chmod +x /usr/local/bin/docker-compose 4- verify that the docker-compose installation was successful by checking the version docker-compose -v This will print out the version we installed: 5- Run a git clone of Servicebot docker deployment git clone https://github.com/service-bot/servicebot-deploy.git 6- git clone command above should create a new directory inside your current working directory. Move into the newly created folder: cd servicebot-deploy/ There should be a file in this directory called “docker-compose.yaml”. This file obtains the configuration of your Servicebot container. 7- [Optional] Configure the docker yaml file. docker If you have your SSL cert and mail server, go ahead and add them to this .yaml file, if not, skip to step #8. Let’s modify this file to add your SSL cert and mail server (you can use any other editor you would like, I used nano) nano docker-compose.yaml For SMTP mail server, edit and uncomment the following lines to match your SMTP server information: SMTP_HOST : "localhost"SMTP_USER : "postmaster@localhost"SMTP_PASSWORD : "password"SMTP_PORT : "587" For SSL: uncomment the two SSL lines and change to the path on your server that you created for the certificates: /path/to/ssl/certs/on/server CERTIFICATES: "./ssl/"  - /path/to/ssl/certs/on/server:/usr/src/app/ssl 8- Build the docker images docker-compose build 9- Start the docker container docker-compose up -d Your new Servicebot website should be live now. Open your browser, go to your server IP address or URL and you should be presented with the following page: 10- That’s it! Initial Servicebot open source setup page Setup Servicebot Once you have installed your Servicebot website, it is time to connect it to your Stripe account and add your business information. You can either use your live or test keys to connect to your Servicebot website. If you haven’t configured your SSL yet, you can only use the test mode keys. Updating Stripe keys in the future is very simple through the Servicebot dashboard. 11- Login to Stripe, navigate to the API link on the left hand side, or simply go to this link . Simply paste your Stripe keys to your Servicebot setup page and then press next. 12- Once your Stripe keys are configured, you should see the following page: Enter an email and password for your administrator user. Then enter your business information and . press submit You are all done. You now have a brand new Servicebot website that you can use to manage your customers, services, and automate your billing. Congratulations! Initial Servicebot page after setup completion Alternative installation methods If you don’t want to go through the effort of installing your own Servicebot server, you could just signup for one through our automated system on the homepage. Just input your email, password, and business name, and you will have a new Servicebot website in less than 20 seconds. ServiceShop Next steps In the future posts, I will focus on Servicebot’s features and use-cases and teach you how to use the platform. If you have any questions or would like to talk to our team, you can email us at or just use our website instant chat. team@servicebot.io

Install and configure an open source CRM for your XaaS business

Docker is awesome — more and more people are leveraging it for development and distribution. Instant environment setup, platform independent apps, ready-to-go solutions, better version control, simplified maintenance: Docker has a lot of benefits. But when it comes to data science and deep learning, there is a certain hitch. You have to memorize all those docker flags to share ports and files between host and container, create unnecessary scripts and deal with CUDA versions and GPU sharing. If you have ever seen this error, you know the pain: run.sh $ nvidia-smi Failed to initialize NVML: Driver/library version mismatch Our goal The purpose of this small post is to introduce you a sufficient set of Docker utilities and GPU-ready boilerplate we often use in our company. So, instead of this: docker run \--rm \--device /dev/nvidia0:/dev/nvidia0 \--device /dev/nvidiactl:/dev/nvidiactl \--device /dev/nvidia-uvm:/dev/nvidia-uvm \ You will end up with this: doc up Cool, right? What do we actually want to achieve: Manage our application state (run, stop, remove) using one command Save all those run flags to a single configuration file we can commit to a git repo Forget about GPU driver version mismatch and sharing Use GPU-ready containers in production tools like Kubernetes or Rancher So here is the list of tools we highly recommend for every deep learner: 1. CUDA First, you will need . It’s an absolute must-have, if you plan to train models yourself. We recommend to use installer type instead of , because it won’t mess your dependencies in future updates. CUDA toolkit runfile deb (Optional) How to check if it works: cd /usr/local/cuda/samples/1_Utilities/deviceQuerymake./deviceQuery # Should print "Result = PASS" 2. Docker You don’t want to pollute your computer with tons of libraries and be afraid of broken versions hell. Also, you won’t have to build and install stuff yourself — usually, software is already built for you and packed in image! is simple: Installing Docker curl -sSL | sh https://get.docker.com/ 3. Nvidia Docker A must have from NVIDIA if you use Docker — it really simplifies using GPU inside Docker containers. utility Installation is really simple: wget -P /tmp https://github.com/NVIDIA/nvidia-docker/releases/download/v1.0.1/nvidia-docker_1.0.1-1_amd64.debsudo dpkg -i /tmp/nvidia-docker*.deb Now, instead of sharing nvidia devices every time like this: docker run --rm --device /dev/nvidia0:/dev/nvidia0 --device /dev/nvidiactl:/dev/nvidiactl --device /dev/nvidia-uvm:/dev/nvidia-uvm you can use a command: nvidia-docker nvidia-docker run --rm nvidia/cuda nvidia-smi Also, you can stop worrying about driver version mismatch: docker plugin from Nvidia will solve your problems. 4. Docker Compose Super useful utility that allows you to store configuration in a file and manage application state more easily. Though it was designed to “compose” multiple docker containers together, docker compose is still very useful when you only have one service. Pick the stable version : docker run here curl -L https://github.com/docker/compose/releases/download/1.15.0/docker-compose-`uname -s`-`uname -m` > /usr/local/bin/docker-composechmod +x /usr/local/bin/docker-compose 5. Nvidia Docker Compose Unfortunately, Docker Compose doesn’t know that Nvidia Docker exists. Lucky, there is a solution: a tiny that generates configuration with driver. Install it using pip: Python script nvidia-docker pip install nvidia-docker-compose Now you can use command instead of . nvidia-docker-compose docker-compose Alternative If you don’t want to use , you can . Just add those options to your : nvidia-docker-compose pass volume-driver manually docker-compose.yml # Your nvidia driver version herevolumes:nvidia_driver_375.26:external: true...volumes:- nvidia_driver_375.26:/usr/local/nvidia:ro 6. Bash aliases But is 21 characters to type! That’s too much. nvidia-docker-compose Lucky we can use bash aliases. Open (sometimes ) in your favorite editor and type those lines: ~/.bashrc ~/.bash_profile alias doc=' 'alias docl='doc logs -f --tail=100' nvidia-docker-compose Update your settings by running . source ~/.bashrc Start a TensorFlow service Now we are ready to use benefits from all those stuff above. For example, let’s run a Tensorflow GPU-enable Docker container. In a project directory create file with the following content: docker-compose.yml version: '3' services:tf:image: gcr.io/tensorflow/tensorflow:latest-gpuports:- 8888:8888volumes:- .:/notebooks Now we can start TensorFlow Jupiter with a single command: doc up is an alias for — it will generate modified configuration file with correct and then run . doc nvidia-docker-compose nvidia-docker-compose.yml volume-driver docker-compose You can manage your service using the same command: doc logsdoc stopdoc rm# ...etc Conclusion But is it worth the effort? Let’s weigh the pros and cons here. Pros Forget about GPU device sharing You don’t have to worry about Nvidia driver version anymore We got rid of command flags in favour of clean and plain configuration No more flag to manage container state --name Well-known documented and widely used utilities Your configuration is ready for orchestration tools like Kubernetes that understand docker-compose files Cons You have to install more tools Is it production-ready? Yep. In our movies recommendation service we use GPU-accelerated TensorFlow network to calculate real time film selection based on user input. Movix We have three computers with Nvidia Titan X in behind Proxy API. Configuration is stored in regular files: because of that it’s really ease to setup development environment or deploy application on a new server. So far it works perfect. Rancher cluster docker-compose.yml Be prepared for the future of ML! If you have any questions or comments, feel free to write here or on twitter @deepsystemsru .

Stories, News & Mentions about GETDUCK

Getduck’s Stories on HackerNoon