Dealpath is the leading cloud-based deal management platform for real estate investment and development teams

Dealpath

dealpath

Artificial Intelligence tools such as Bard, ChatGPT, and Bing Chat are the current big names in the category which is on the rise. Large Language Model (LLM) LLMs are trained on vast data sets to be able to communicate by using everyday human language as a chat prompt. Given the flexibility and potential of LLMs, companies are integrating them into many workflows inside the tech industry to make our lives better and easier. One of the main AI-workflow integrations is an auto-complete code plugin, providing the ability to foresee and generate matching code patterns in order to code faster and more efficiently. In the current market of AI code-generator tools, there are many players, including , , , , and more. This blog post outlines examples of common security pitfalls that developers face while using such tools. GitHub Copilot Amazon CodeWhisperer Google Cloud Code (Duet AI) Blackbox AI Code Generation The goal of this article is to raise awareness and emphasize that auto-generated code cannot be blindly trusted, and still requires a security review to avoid introducing software vulnerabilities. Note: some vendors suggest that their auto-complete tools may contain insecure code snippets. Many articles have already emphasized that auto-complete tools generate known vulnerabilities such as , SQL injection, and XSS. This post will highlight other vulnerability types that depend more on the code’s context, where there is a high probability for AI auto-complete to slip bugs into your code. IDOR AI models trained for code are usually trained on large code bases with the main mission of producing functioning code. Security and AI Code Generating Tools Many security issues have a known and defined way of mitigation without compromising the performance side (e.g. SQL injections could be avoided by not concatenating user input/parameters directly into the query) - and therefore, can be eradicated as there is no reason to write the code in an insecure manner. However, many security issues are which could be perfectly safe when implemented as internal functionality, while when facing external input from a client, become a vulnerability, and are therefore hard to distinguish through the AI’s learning dataset. context-dependent In such vulnerabilities, the specific usage of the code usually depends on other parts of the code and the application’s overall purpose. Here are some common use cases and examples that we tested, showcasing some of these types of vulnerabilities: Use Case: Fetching Files from User Input - Arbitrary File Read Use Case: Comparing Secret Tokens - Type Juggling/Coercion Use Case: Forgot Password Mechanism - Unicode Case Mapping Collisions Use Case: Generating Configuration File - Bad Security Practices Use Case: Configuration Objects - Inconsistencies Leading to Insecure Deserialization After seeing how the AI code generation tools handle the above cases, we tried to test their ability to create proper security filters and other mitigation mechanisms. Here are some common use cases and examples that we tested by deliberately requesting secure code: Secure Code Request Use Case: Reading Files - Avoiding Directory Traversal Secure Code Request - File Upload - Restricted Extensions List Use Case: Fetching Files from User Input - Arbitrary File Read Many applications include code that fetches a file back to the user based on a filename. Using directory traversal for reading arbitrary files on the server is a common method bad actors use to gain unauthorized information. It may seem obvious to sanitize file name/file path input coming from the user before fetching the file back, but we assume it is a hard task for an AI model trained on generic codebases -  this is because some of the datasets as part of their functionality (it may even downgrade performance or harm functionality) or are not harmed by unsanitized path as they are intended to be only for internal usage. do not need to use sanitized paths Let’s try to generate a basic file fetching function from user input, using Google Cloud Code - Duet AI auto-complete. We will let Duet AI auto-complete our code, based on our function & route names - AI Tool Suggestion (Google Cloud Code - Duet AI): As we can see in gray, the autocomplete suggestion is to use Flask’s “ ” function, which is Flask’s basic file-handling function. send_file But even Flask’s documentation states “ ”, meaning that while inserting user input directly to the “send_file” function the user will have the ability to read any file on the filesystem, for example by using directory traversal characters such as “../” in the filename. Never pass file paths provided by a user Proper Implementation: Replacing the “send_file” function with Flask’s safe “send_from_directory” function will mitigate the directory traversal risk. It will only allow fetching files from a specific hard-coded directory (“exports” in this case). Use Case: Comparing Secret Tokens - Type Juggling/Coercion In some programming languages, such as PHP and NodeJS, it is possible to make comparisons between two different types of variables and the program will do a behind the scenes to complete the action. type conversion Loose comparisons do not check the variable’s type behind the scenes and therefore are more prone to type juggling attack. However, the use of loose comparisons is not considered an insecure coding practice by itself - it depends on the context of the code. And again, it is difficult for AI models to tell the cases apart. The most common example of PHP type juggling is loose comparisons of secret tokens/hashes where user input which is read via a JSON request (which enables control of the input’s type) reaches a loose comparison (“ =”). AI Tool Suggestion (Amazon CodeWhisperer): ”) instead of a strict one (“ As it appears, CodeWhisperer generates loose comparison conditions in auto-complete suggestions: The loose comparison of the secret_token allows an adversary to bypass the comparison $data[“secret_token”] == “secret_token”. While submitting a JSON object with the value of “secret_token” being True (boolean), the (even on newer PHP versions). loose comparison result is True as well Even when “hinting” the tool (using a comment) to write the check “securely”, it did not generate a code snippet containing a strict comparison - Proper Implementation: Only when specifying a strict comparison (“===”) we are protected against type juggling attacks. Use Case: Forgot Password Mechanism - Unicode Case Mapping Collisions Vulnerabilities in the “Forgot Password” mechanism are very common in SaaS applications and were the root cause behind CVEs such as (Wordpress), (Django), and (GitLab). CVE-2017-8295 CVE-2019-19844 CVE-2023-7028 Specifically, a Unicode Case Mapping Collision vulnerability occurs when user input is either uppercased or lowercased in a comparison expression while its original value is also used in the code. Some different characters will result in the same char code when transformed. For example, “ß” and “SS” will both transform into “0x00DF” when uppercased. Such cases are usually used to bypass/mislead some security checks by tricking the comparison check and later using the original value which is different than expected. These cases are quite elusive to grasp, especially when many implementations are possible. A mechanism that’s particularly susceptible to this vulnerability type is the . It is common to normalize the user’s input into uppercase or lowercase (in email addresses or domain names) when performing the check in order to avoid unwelcome mismatches. AI Tool Suggestion (GitHub Copilot): forgot password mechanism As an example, taking a look at the Copilot-generated code for the forgot password function (below) shows that it is vulnerable to this issue. Copilot’s code first looks up the lowercased email address: Then, when following through with the rest of the process it suggests the following: As we can see, the autocomplete suggestion generates a random password and sends it to the user’s email address. However, it initially uses a lowercase version of the user’s email for fetching the user account, and then continues to use the original user’s email ‘as-is’ (without a lowercase conversion) as the target of the recovery email. For example, lets say the attacker owns the inbox for “miKe@example.com” (the “K” is a unicode character) and uses this email for the “Forgot Password” mechanism. The emails “miKe@example.com” and “mike@example.com”, are equivalent ‘as-is’, but after lowercase conversion they will be - Kelvin Sign not Using the email address “miKe@example.com” will fetch the account information for “mike@example.com” but the reset password will be sent to the attacker’s inbox (“miKe@example.com”), allowing takeover of the “mike@example.com” account. The email address used for fetching the user account must exactly equal the recovery email address (i.e. send_email parameter will also use email.lower()). Proper Implementation: In addition, as a precaution, it is recommended to use the value that was already stored in the server instead of the one supplied by the user. Use Case: Generating Configuration File - Bad Security Practices Another topic that could confuse auto-complete tools is writing configuration files, especially for complicated cloud infrastructures. The main reason is that indeed there are security best practice guidelines, but not everybody follows them, and in some cases, it is needed to go against them. Such code samples might find their way into the AI training dataset.  As a result, some auto-complete outputs will violate the recommended security guidelines. In the following example, we will create a configuration file for a , the smallest execution unit in Kubernetes. Kubernetes Pod In this example, we start creating a Pod configuration file. AI Tool Suggestion (Blackbox AI Code Generation): The suggestion creates the capabilities section and adds a Linux kernel capability (SYS_ADMIN) to the container which is essentially equivalent to running the pod as the root user. Is it better to just omit the securityContext then? No - since many other capabilities will then be added by default, violating the . Proper Implementation: principle of least privilege According to , the most secure setup is to first drop all Linux kernel capabilities and only afterward add the ones required for your container. Docker security best practices In order to fix the issues mentioned above, the capabilities section begins by dropping all capabilities, and will then gradually add the needed ones for our container. Use Case: Configuration Objects - Inconsistencies Leading to Insecure Deserialization Many vulnerabilities require defining a configuration object, which decides how the application will handle the required component. The example we chose is deserialization library in C#, which can be used securely or insecurely depending on the JsonSerializerSettings object’s configuration, and specifically, the . Newtonsoft’s JSON TypeNameHandling While we tested the deserialization code, we noticed that the configuration object’s definition is quite random, and subtle changes can lead to generating a different configuration set. The following examples display an inconsistent behavior that is based only on the method names used by the developer: AI Tool Suggestions (Amazon CodeWhisperer): We can see two different configurations that both result in insecure JSON deserialization due to the TypeNameHandling property being “Auto” and “ALL”. These properties allow the JSON document to define the deserialized class - letting attackers deserialize arbitrary classes. This can be easily leveraged for remote code execution, for example by deserializing the class “System.Diagnostics.Process”. The only difference in the developer’s original code are the method names. Proper Implementation: By default, a JsonSerializerSettings object is instantiated with “TypeNameHandling = TypeNameHandling.None”, which is considered to be secure. Thus, we use JsonSerializerSettings’s default constructor which will result in a safe TypeNameHandling value. Similar to the first use case, where we examined the vulnerable code produced by Copilot for fetching files, here we attempt to request a secure version of a file reading mechanism. Secure Code Request Use Case: Reading Files - Avoiding Directory Traversal AI Tool Suggestion (GitHub Copilot): We can see that the security check is really naive, only avoiding dot-dot-slash sequences that are needed for conducting directory traversal attempts. The path parameter can be an absolute path - pointing to any desired path on the system (for example, /etc/passwd), which defeats the purpose of the security check. Proper Implementation: Here, the secure_file_read function gets a (relative) filename parameter along with a directory (safe_dir) where the filename should reside (and which should not be escaped).
It creates an absolute path by joining safe_dir and filename and proceeds to get its canonicalized form by calling realpath. It then gets the canonicalized form of the safe_dir folder. Then, the file contents are returned only if the canonicalized path starts with the canonicalized safe_dir. It is a common use case to only accept specific file types for upload based on their extension. Secure Code Request - File Upload - Restricted Extensions List In the following example, we asked the AI auto-completer to generate a function tasked with filtering dangerous file extensions. AI Tool Suggestion (Blackbox AI Code Generation): The suggested Python code takes the filename, separates the extension, and compares it against a list of dangerous extensions. At first glance, this code snippet looks secure. However, forbid filenames ending with a dot, and when supplying a filename ending with a dot (“.”) during file creation, the dot will be discarded. Meaning, that the generated filter could be bypassed on Windows when submitting a file with a malicious extension followed by a dot (ex. “example.exe.”). Windows filename conventions Usually the better approach is to use a , which checks the file extension only against allowed extensions. However, since the tools took a blacklist approach (which is needed in some cases) we will outline a safer blacklist implementation: Proper Implementation: whitelist In the following snippet, we strip away as much control the user had on the input by stripping the extension from all non-alphanumeric characters and concatenating it to a newly generated filename. Automatic co-programmers are great for suggesting ideas, auto-completing code, and speeding up the software development process. We expect these tools to keep evolving as time goes on, but for now as demonstrated with the use cases in this post, auto-complete AI solutions have many challenges to overcome until we can ease our minds and trust their output without double-checking for bugs. Bottom line - use with caution A possible way to reduce the likelihood of introducing vulnerabilities is deliberately requesting the AI code generation tool to generate secure code. Although this may be useful in some use cases, it is not foolproof - the “seemingly secure” output should still be reviewed manually by someone proficient in software security. To help secure your software, we recommend both manually reviewing code produced by AI generative tools, as well as incorporating security solutions such as Static Application Security Testing (SAST) which can trustfully discover vulnerabilities as you go. As seen in the example above, SAST tools can alert and catch the unicode case mapping collision described in use case #3. This proactive approach is essential to ensuring the security of your software. JFrog Security Research Team Recommendations The security research team’s findings and research play an important role in improving the JFrog Platform’s application software security capabilities. Stay up-to-date with JFrog Security Research Follow the latest discoveries and technical updates from the JFrog Security Research team on our , and on X . research website @JFrogSecurity

Analyzing Common Vulnerabilities Introduced by Code-Generative AI

কোড-জেনারেটিভ এআই দ্বারা প্রবর্তিত সাধারণ দুর্বলতাগুলি বিশ্লেষণ করা

Analyse häufiger Schwachstellen, die durch Code-generative KI entstehen

Análisis de vulnerabilidades comunes introducidas por la IA generativa de código

Analyse des vulnérabilités courantes introduites par l'IA génératrice de code

कोड-जेनरेटिव एआई द्वारा प्रस्तुत सामान्य कमजोरियों का विश्लेषण

コード生成 AI によってもたらされる一般的な脆弱性を分析する

코드 생성 AI로 인해 발생하는 일반적인 취약점 분석

Analisando vulnerabilidades comuns introduzidas pela IA geradora de código

Анализ распространенных уязвимостей, создаваемых кодогенерирующим ИИ

Kod Üreten Yapay Zekanın Getirdiği Yaygın Güvenlik Açıklarını Analiz Etme

Phân tích các lỗ hổng phổ biến được giới thiệu bởi AI tạo mã

分析代码生成人工智能引入的常见漏洞

Sometimes, you can't simply open your in Chrome to test your website. You need the features of a real web server like NGINX. index.html A common case where directly loading falls short is trying to serve local files using the Fetch API. You run into Cross-Origin Resource Sharing (CORS) errors. I hit this little bump when creating the example for this . In addition to preventing CORS errors, it's more realistic to use a production web server in your development environment. index.html blog post about the ClipBoard API To get started, you only need to and run one command. install Docker $ docker run -d --name nginx-development -v $(pwd):/usr/share/nginx/html -p 80:80 nginx The above command does a few things. The flag ensures that you don't get stuck attached to the container. Using the keyword argument gives this container instance a name, so we can reference it later. The maps our local machine's port to the container port that NGINX uses to serve HTTP. This port-mapping allows us to go to to see our development site. -d --name nginx-development -p 80:80 80 http://localhost The keyword argument needs a little more explaining. It links our current working directory to a directory the running container instance. This directory in the container is the default location where NGINX looks for files to serve. If you want to serve a different directory, then you need to change the argument. For example, I like to put my at in my project directory. To adjust for that, my argument would look like . -v $(pwd):/usr/share/nginx/html in --volume index.html ./src/index.html -v -v "$(pwd)/src:/usr/share/nginx/html" If you use your browser to navigate to , you should be able to see your site. But if you make changes to your or any other files on your site, refreshing your browser won't show them. To reload the site after you make changes, use the following command. http://localhost index.html $ docker restart nginx-development It's good that we can reload our changes, but it requires the extra step: running the above command. With the previous workflow – opening up directly – we only needed one step: refreshing our browser. Let's fix that 😤 docker restart index.html Writing an Auto-Reload Script To create this script, we need to either Poll the directory every so often and keep track of changes ourselves Have some way to be notified when there is a change To avoid managing our own state and to keep our bash script simple, let's go with option 2. How can we get these notifications? On Linux, there is a command-line tool called that relies on the underlying . We could use this, but then we would be locked-in to Linux. I develop on Mac OS and Linux, so I'd like to support both – even better if we can support Windows too. inotifywait inotify C API There is another tool called that works on Mac OS, Linux, and Windows. It may not be installed by default on your operating system. On Mac OS, I needed to install using the following command. fswatch fswatch $ brew install fswatch Now let's see how this tool works. Let's try running the tool and pointing it at our source directory. $ fswatch src My console just hangs with a blinking cursor. Let's see what happens if we modify one of the files in the directory. I'll try editing one of my css files... src $ fswatch src
/Users/curtis/git/licorice-works/src/styles/4913
/Users/curtis/git/licorice-works/src/styles/projects.css
/Users/curtis/git/licorice-works/src/styles/projects.css
/Users/curtis/git/licorice-works/src/styles/projects.css~ Wow 😲 Almost immediately, gives us the above output with name of the file that I modified. fswatch This is a lot of change notifications for modifying one file though. It looks like we don't want notifications for the file or for the file. These seem to be oddities of how saving files works on Mac OS. Luckily, the tool has some arguments that we can use to filter the change notifications. By default, it includes everything. This behavior makes including exactly the file types we want a little weird. First, we need to use to exclude everything. Then we can use to re-include only files that end in . With our new command, we only get notifications for the CSS file that we changed – not those other weird files. 4913 .css~ fswatch -e '.*' -i 'css$' css $ fswatch -e '.*' -i 'css$' src                     
/Users/curtis/git/licorice-works/src/styles/projects.css
/Users/curtis/git/licorice-works/src/styles/projects.css This is still two notifications though. We only want to reload NGINX once per change. Thankfully, has a flag that we can use to batch the changes together. fswatch --one-per-batch $ fswatch -o -e '.*' -i 'css$' src
2 After adding , we only get one line of output when we modify a file. Let's use this output to trigger a restart of our Docker container. -o $ fswatch -o -e '.*' -i 'css$' -i 'html$' -i 'js$' src | \
while read -r line; do docker restart nginx-development; done That command looks more complicated than the commands we've used so far. Let's dig in. First, I added two more include filters. One to include any updates we make to files and another to include files. Then we pipe the output of this command into a while loop. If you're not familiar with pipe, it just sends the output of the first command as input to the second command. The second command that we're piping into is a while loop. The general syntax of a while loop looks like the following. -i .html .js | while <condition>;
do
  <command to execute>
done It's a little easier to read when it's not squished onto one line. For the part, we used a command . Whenever there is a line of input to read, this command reads it, stores it in the variable , and returns true. In our script, this means whenever prints out a batch of changes, our loop will execute once. So each time there is a batch of changes, we will execute our restart command which is the same as above . <condition> read -r line line fswatch docker restart nginx-development You could get started by running the two commands. $ docker run -d --name nginx-development -v $(pwd):/usr/share/nginx/html -p 80:80 nginx
$ fswatch -o -e '.*' -i 'css$' -i 'html$' -i 'js$' $(pwd) | while read -r line; do docker restart nginx-development; done But this is a lot to remember or to copy-paste each time, so I wrote a script that takes only one argument: your source directory. You can copy this script and easily reuse it for each of your projects. nginx-autoreload.sh #!/bin/bash

CONTAINER_NAME="nginx-development"

# Use the first argument as our source dir.
# Default to current working directory if no argument is supplied.
SOURCE_DIRECTORY=$1
if [ ! $1 ]; then
    SOURCE_DIRECTORY="$(pwd)";
    echo "No source directory provided. Defaulting to current working directory: ${SOURCE_DIRECTORY}";
else
    # Get the full path
    SOURCE_DIRECTORY=$(realpath "${SOURCE_DIRECTORY}")
fi

docker rm --force "${CONTAINER_NAME}" &> /dev/null;

docker run -d --name "${CONTAINER_NAME}" -v "${SOURCE_DIRECTORY}":/usr/share/nginx/html -p 80:80 nginx > /dev/null;
if [ $? -ne 0 ]; then
    echo "Failed to initialize container";
    exit 1;
else
    echo "Successfully started container: ${CONTAINER_NAME}";
fi

echo "Waiting for changes..."
fswatch -o -e '.*' -i 'css$' -i 'html$' -i 'js$' "${SOURCE_DIRECTORY}" |
while read -r line; do
    docker restart "${CONTAINER_NAME}" > /dev/null;
    if [ $? -eq 0 ]; then
        echo "Successfully reloaded ${CONTAINER_NAME}";
    else
        echo "Failed realoading ${CONTAINER_NAME}";
    fi
done You can use the script like , and it will take care of setting up the container and watching for changes. $ ./nginx-autoreload.sh <your-source-directory Note: remember to install Docker and fswatch first! We did it! We wrote a single script to host and auto-reload your development site with NGINX 🥳 Also published . here

How to Use NGINX to Host a Local Site Using Docker

Gone are the days where every IT guy/girl/person had a specific job like QA, developer, architect etc.  Good riddance, since we all are creative and want to have fun and indulge in more than we can handle!  Welcome to the "full-stack" developer, whom, once you hire, you hold them responsible and heave a sigh of relief.. This article addresses a specific way of and should be of interest to the 'full-stack' developer. developing a web application In today's normal design, a web application follows the same paradigm as client-server application. The client handles the 'front-end' user interactions and is mostly concerned with the user interface. The server is the 'back-end' and services requests from the front-end.  One challenge here is how to coordinate the development of both so that they work with each other, and manage the complexities. Such complexities include different languages for server(python)/client (javascript), configuration files that are interdependent, settings of middle-ware such as nginix, handling the git repositories for both and more. As a lone-wolf full-stack developer having to do everything, we need some cogent method. What if we do this? Have a single repository with the server and client code in separate folders. This allows the developer to be able to view the client code/logic while working on the server code and vice-versa. This reduces the complexity of  interfacing  the client and server, reduces bugs and makes it easy to test.  It also has advantages in deployment.  Very often the web application has a small audience and efforts at scaling are not necessary. Hosting both the server and client in one physical server makes deployment easy. 'Version' mismatch between client and server  is either elminated or minimized with the code for both client and server being in the same repo. Have the server, serve the client at a user request from the browser. The server will also serve client requests. This way the server can control the configuration of the client. Further it opens up an opportunity of centralizing client control, such as limiting the number of clients gracefully, keeping track of resources assigment/usage by each client etc.
It also reduces the number of 'ports' exposed for internet connections. Design concepts The server runs on the backend, and serves ONE request directly from the user, which  is the invoking of the application from the browser.  The response by the server is to serve the client application to the user, which is a SPA (single page application almost always a Javascript application).  The client which runs in the browser, then handles all user interactions and calls the server as necessary to satisfy user requests. The are many components in here - the server, the client, middleware (nginix) and each can be elaborate. Here we concentrate on the glue that binds these together.  This glue is the configuration file. This is diagrammed below: The details of the config which is the core item in this architecture are: Config Item Description port This is port that the webapp runs on. User requests from the browser are routed to this port by nginx url This is the url that nginx routes to the webapp running on port serverUrlPrefix This url prefix is used by the client to make requests to the server. (Note: The client runs on the user's browser) ClientResourcePrefix This prefix is used to send requests to the server to  access resources in the client folder. These are the static resources (images, javascript) of the client client loc Location of the client folder. The server serves the index.html from this location, as well the client's static resources Logical: a. The server is a normal back end server and has its own set of routes PLUS one route that serves the client. b. The client is a separate folder in the repo named 'client'. It is a javascript single page App. The application initiation and interactions are depicted in the flow diagram below. The numbers in the text show the sequence of events: Implementatin An example app that follows this modes is described below. For that demp app, the above logical design is implemented as: Server is implemented Using a python Sanic webserver Client is a Javascript Single Page App using lit-elements. Folder Structure: : This folder contains the server code. Here it is written in Python and uses the Sanic webserver. Server folder This folder contains the client code that runs on the browser. Here it is a Javascript SPA (single page application) and uses the popular web-component framework Lit-element. The folder c is the output of ‘building’ the client app and is the actual folder where the final client code resides. Client folder: lient/build contains the Server and Client folders. This folder can be considered as the git repository. Main Folder: Example: Fill up a form Client Javascript application: Display the form, collect data and send to server Server Python/Sanic Application: Receive form data Nginix App: Route user/client requests/responses Application: Two Screen shots: 1. The form, 2. The server response are: The first screenshot is from the client and shows the form. The ‘SaveMyForm’ button causes the client to invoke the server with an api using the ‘serverUrlPrefix’.  The second screenshot shows the response: a picture (a client resource) with a message from the Server at the bottom. Steps to implement the example: Create the config file: Develop the client:
a. Index.html with placeholder for the config client related parameters
b. The form page component (use lit element for convenience) Develop the server:
a. Read config file and use it to modify client index.html
b. Have a route to serve the client (with modified index.html of step a)
c, Have a route to receive the form data The relevant code (config.yml, nginx, server app, client-index.html, form.js).  Of these the config.yml contains the core part of this article. It is read, used and distributed to the client by the server app. In the example, the server app distributes the config to the client, by modifying the index.html. The entries in the config file are also distributed to the nginx config file via 'hardcoding'. The entries in the config file for this example are: Config Item value Description port 8002 This is port that the backed server of the webapp runs on.  This port value and the url below are used in the nginx config file for routing. url localhost/diy This is the url that nginx routes to the webapp running on port. The user enters this url in the browser to access the application serverUrlPrefix localhost/diy This url prefix is used by the client to make requests to the server. Note: The client runs on the user's browser. Any request with this prefix is routed to the server app. ClientResourcePrefix spa This prefix is used to send requests to the server to  access resources in the client folder(see Client loc below). These are the static resources (images, javascript) of the client. The server uses the clientLoc (see below) to serve these requests. clientLoc client/build Location of the client folder. The server serves the index.html from this location, as well as other client's resources The other related code is listed below. Hopefully these are self explanatory given some familiarity with the language/tools used. This is a yaml file containing the items described above: Config file: port: 8002  
url: 'localhost/diy'
serverUrlPrefix: 'localhost/diy'
clientResourcePrefix: 'spa'
clientLoc: 'client/build' : (note the values from the config file are hardcoded here) Nginx config file server {
        listen 80 default_server;
        listen [::]:80 default_server;

        root /var/www/html;

        # Add index.php to the list if you are using PHP
        index index.html index.htm index.nginx-debian.html;

        server_name _;

        location / {
                # First attempt to serve request as file, then
                # as directory, then fall back to displaying a 404.
                try_files $uri $uri/ =404;
        }
    set $diyPort 8002;
    location ~ /diy/(.*) {
        #pass the query params also!
        proxy_pass  http://0.0.0.0:$diyPort/$1$is_args$args;
    }
    location /diy {
       return 302 /diy/;
    }
} This python module serves the client application at the ‘/’ route. It also services the client request ‘saveform’ (see the Application Flow diagram earlier in this article): Server code: from sanic import Sanic, response
import json
import yaml

import os
installDir = os.path.dirname(os.path.realpath(__file__))

app = Sanic(name='DiyArticle')
#read config file
def getConfig():
    configFileNm = f'{installDir}/config.yml'
    with open(configFileNm, "r") as f:
        G = yaml.safe_load(f.read())
    print(f'[global]ConfigFileName: {configFileNm}\nConfig:\n{json.dumps(G,indent=4)}')
    return G

G=getConfig()

clientDir = f'{installDir}/../{G["clientLoc"]}'
baseUrl = f'{G["clientResourcePrefix"]}'
#clientDir = '/home/vv/git/ASrvrClntIn1/client/deploy'

app.static(f'/{G["clientResourcePrefix"]}', clientDir)


@app.route("/saveform",methods=['POST'])
async def test(request):
    return response.text("Your Form is saved on the Server!")

@app.route("/")
async def clientSPA(request):
    with open(f'{clientDir}/index.html','r') as f:
        indexHtml = f.read()
    #modify and insert the baseUrl and the serverUrlPrefix (to call server api from client)
    indexHtml = indexHtml.replace('${baseUrl}',baseUrl)
    indexHtml = indexHtml.replace('${serverUrlPrefix}',G["serverUrlPrefix"])
    print(f'Vasan:\n{indexHtml}')

    return response.html(indexHtml)

if __name__ == "__main__":
    app.run(host="0.0.0.0", port=G['port']) i (note the ‘${..}’ templated entries that are filled from the config file). This is the SPA (single page javascript app) and uses the custom component ‘diy-form’ Client code: ndex.html <!DOCTYPE html>
<html>
<head>
    <base href="${baseUrl}/" target="_blank">
    <script type="module" src="form.js"></script> 
</head>
<body>
    <div>
        <diy-form server-url-prefix="${serverUrlPrefix}""></diy-form>
    </div>
</body>
</html> - The custom component written using the Lit element framework. This is the file.  Refer to the Lit element documentation for elaborate details ( Client code: diy-form form.js import {LitElement, html,css } from 'lit';
import {repeat} from 'lit/directives/repeat.js';

async function postDataToUrl(url = "", data = {}) {
  // Default options are marked with *
  const response = await fetch(url, {
    method: "POST", // *GET, POST, PUT, DELETE, etc.
    mode: "cors", // no-cors, *cors, same-origin
    cache: "no-cache", // *default, no-cache, reload, force-cache, only-if-cached
    credentials: "same-origin", // include, *same-origin, omit
    headers: {
      "Content-Type": "application/json",
      // 'Content-Type': 'application/x-www-form-urlencoded',
    },
    redirect: "follow", // manual, *follow, error
    referrerPolicy: "no-referrer", // no-referrer, *no-referrer-when-downgrade, origin, origin-when-cross-origin, same-origin, strict-origin, strict-origin-when-cross-origin, unsafe-url
    body: JSON.stringify(data), // body data type must match "Content-Type" header
  });
  if (response.status != 200) {
    return `Status: ${response.status} : ${response.statusText}`
  }
  return await response.text()
}

class FormElement extends LitElement {
  static styles = css`
    input.invalid {
      border: 1px solid red;
    }
  `;

  constructor() {
      super();
      this.saveDataUrlSuffix='saveform'
      this.state='open'
      this.serverUrlPrefix = this.getAttribute('server-url-prefix')
  }

  async receiveConfigInfo(config,parentObj) {
    console.log(`Config info received in brRegForm: ${JSON.stringify(config)}`)
    this.parent=parentObj
    this.config = config

  }


  getFormData(myForm) {
    let result = (new FormData(this.form)).entries()
    let fd={}
    for (const element of result) {
      console.log(element);
      if (fd.hasOwnProperty(element[0])) 
        fd[element[0]].push(element[1])
      else fd[element[0]]=[element[1]]
    }
    //include value='' for radio buttons which were left blank
    let rbs=myForm.querySelectorAll('[type="radio"]')
    for (const element of rbs) {
      if (fd.hasOwnProperty(element.name)) continue;
      else fd[element.name]=['']
    }
    return fd
  }

  validateFormData(fd) {
    return ''
  }

  async submit(e) { 
    console.log('Form data')
    this.form = this.shadowRoot.querySelector('form')
    let fd = this.getFormData(this.form)
    console.log(fd)
    let errMsg = this.validateFormData(fd)
    if (errMsg) {
      let elmt = this.shadowRoot.querySelector("errMsg")
      elmt.innerText=errMsg
      elmt.style.display='block'
    } else {
      // await this.parent.setChart(fd);
      //this.parent.closeApp()
      let urlToPost=`http://${this.serverUrlPrefix}/${this.saveDataUrlSuffix}`
      console.log(`Form data submitted to ${urlToPost}`)
      this.serverMsgresp = await postDataToUrl(urlToPost,fd)
      if (this.serverMsgresp.includes('saved')) this.state='submitted'

    }
    this.requestUpdate()
  }


  render() {
    if (this.state=='submitted')
    return html`<img src="img/HalloweenThumbsUp.avif" /><br/>
      <span style="font-size: 200%;color: blue;">${this.serverMsgresp}</span>`
    return html`
      <form onsubmit="return false;">
      <div width="80%">
      <div>
      <h1>Form to be filled and submitted to Server</h1>

      Please fill out the following details and click on the 'I agree' button. <br/><br/>
      <b>Choose Like-NotSure: </b>
      <div id="session" class="flex-container">
        <label>
        <input type="radio" id="session1" name="session" value="session1">
        I like this article..
        </label>
        &nbsp;&nbsp;
        <label>
        <input type="radio" id="session2" name="session" value="session2">
        Not exactly sure..
        </label>
      </div>
      <br/>
      <b>contact detail:</b>
      <div class="flex-container">
        <label>Name in full: <input  id="Name" name="Name" style="width: 15em;"/></label>
        <label>Email: <input  id="Email"  name="Email" style="width: 15em;"/></label>
        <label>Phone #: <input  id="Phone" name="Phone" style="width: 10em;"/></label>
      </div>
      <b>Familiarity with both Python and javascript web components:</b>
      <div id="pranayama" class="flex-container">
      <label><input type="radio" name="pranayama" value="None">Not at all</label>
      &nbsp;&nbsp;
      <label><input type="radio" name="pranayama" value="somewhat">Somewhat (occasionaly develop in these)</label>  &nbsp;&nbsp;
      <label><input type="radio" name="pranayama" value="expert">Expert (regular and frequent practioner)</label> 
    </div>
    <br/>

    </div>
    </form>
    <errMsg id="errMsg" style="background-color:red; display=None;"></errMsg>

    <button id="save" style="background-color:lightgreen;" @click="${this.submit}">SaveMyForm</button>
    `
  }
}
customElements.define('diy-form', FormElement); Apologies for a long article, but I am sure that the effort in the reading will be rewarded! Also published here.

Learn To Build a DIY Web App

How some desired Jesus to mix the New Law with the Old Law; and concerning the legion of swine;

Adding Serverless Authentication Microservice to a HTML, CSS and Javascript App

🏆 What will you learn in this tutorial? Single page application fetches and loads the main UI skeleton into the browser only once, during the first fetch. And thereafter, loads the subsequent UI modules dynamically without re-fetching as the user interacts with the app. Look at the example given below. It is a simple single-page application developed using purely HTML, CSS and Javascript. In this tutorial, you are going to learn how to implement it. Open Example In New Tab 💡 What's New In This Tutorial? Front-end frameworks such as React, Angular, Vue, etc. exist today that enable the development of single-page applications. But the development of single-page applications using purely HTML, CSS and Javascript is still not very straightforward. In this tutorial, you will learn about , which allows you to architect HTML applications in a modular fashion. It paves the way for developers to choose just HTML, CSS and Javascript for writing complex apps with a scalable, maintainable and testable codebase. SfNav, a web component ⚠ Before You Begin The following are the pre-requisites for this tutorial: - You need to be familiar with the basic concepts of HTML, CSS & Javascript. You need not be an expert developer, but basic knowledge is necessary. could be a good starting point for this. Familiarity with Web Development W3 Schools Roadmap - In this example, we will use an NPM-based web server. So to follow this tutorial properly, I will recommend that you have npm installed on your development machine. Once you get the hang of it, you can also try the same method on other web servers, no problem! NPM Installation - You will also need a good web editor. I like VS Code. You can use any editor of your choice. Editor 🍽 Step 1 - Project Setup 1.1 Project Directory Create a new folder on your machine for this task. I called it SfNav-Demo. You can name it as you want. 1.2 Initialize NPM npm init Just accept the default values, and keep pressing ENTER until the initialization completes and the package.json file is generated. Your project will now have only one file, namely package.json. 1.2 Create the dist/ folder Create a folder to store all the html files. I called it "dist/". 1.3 Install the HTTP-server packages npm install connect serve-static 1.4 Create index.js Create index.js, to hold the code for the HTTP-server. Paste the code given below in index.js. var connect = require('connect');
var serveStatic = require('serve-static');

connect()
     .use(serveStatic('./dist'))
     .listen(8080, () => console.log('Server running on 8080...')); Your folder structure should look like this: 1.5 Start the HTTP-server Start the web server using the below command node index.js Browse to for confirming that the web server is running. A running server will throw an error message similar to the one given below. This is because we haven't yet created index.html in the dist folder. http://localhost:8080/ 1.6 Create dist/index.html Create index.html file in the dist folder and initialize it with the following code: <html>
hello sfnav
</html> Go back to the browser and refresh it. Confirm that the HTTP-server now serves the index.html page. At this point, the basic project setup is complete. Let us move on to the next steps. 💀 Step 2 - Create The HTML App Skeleton The code of the HTML skeleton is given below. Copy and paste it into your dist/index.html. <!DOCTYPE html>

<html>
  <head>
    <meta charset="utf-8" />
    <meta name="viewport" content="width=device-width, initial-scale=1">
    <title>&lt;sf-nav&gt; Demo</title>
    
    <link href="https://fonts.googleapis.com/icon?family=Material+Icons" rel="stylesheet">
    
    <link rel="preconnect" href="https://fonts.googleapis.com">
    <link rel="preconnect" href="https://fonts.gstatic.com" crossorigin>
    <link href="https://fonts.googleapis.com/css2?family=Source+Sans+Pro:wght@200;300;400;600&display=swap" rel="stylesheet">
    
    <script src="https://unpkg.com/@webcomponents/webcomponentsjs@latest/webcomponents-loader.js"></script>
    
    <script type="module">
        import {LitElement, html, css} from 'https://unpkg.com/lit-element/lit-element.js?module';
        import {SfNav} from 'https://unpkg.com/sf-nav/sf-nav.js?module';
    </script>
    <style>
      
      body {
        background-color: #efefef;
        margin: 0px;
        font-family: 'Source Sans Pro';
      }

      sf-nav:not(:defined) {
        display: none;
      }
      
    </style>
  </head>
  <body style="margin: 0px;">
    <sf-nav>

      
      <h2 slot="brandName"><a href="#home" >Superflows</a></h2>
      
      <a slot="brandImage" href="#home" ><img alt="logo" src="https://superflows-images.s3.ap-south-1.amazonaws.com/superflows_black_transparent_200.png" /></a>
      
    </sf-nav>
  </body>
</html> This skeleton code does the following: It sets up the viewport It includes the material icons package, which is required by the SfNav web component It includes fonts, although you can use any fonts of your choice It includes the web component loader script It includes the Lit library. SfNav component depends on Lit, hence this inclusion is necessary It includes the SfNav web component library It includes a style block that - (1) Sets some basic styling to the document body and (2) Ensures that the SfNav component will show up only after it is defined properly It then includes/renders the SfNav component in the body HTML with a brand name and brand image. 💾 ↺ Save dist/index.html, and reload the browser. You will see that a bare-bones navigation component has loaded which only contains the brand name and brand image. The point to note is that it contains both the header part and the footer part. ⥣ Step 3 - Setup Routing 3.1 Understanding Links SfNav recognizes routes that are prefixed with hash. For example, in the file dist/index.html, the brand name and the brand image are linked to #home, which essentially means that the component home.html will be loaded if the user clicks on either of the brand name and the brand image links. 3.2 Setup Route Container Create a page content slot in the sf-nav block as shown below. This container sits between the header and footer and dynamically displays the page content as the user navigates across pages. <sf-nav>

        
        <h2 slot="brandName"><a href="#home" >Superflows</a></h2>
        
        <a slot="brandImage" href="#home" ><img alt="logo" src="https://superflows-images.s3.ap-south-1.amazonaws.com/superflows_black_transparent_200.png" /></a>

        
        <div slot="content">
        </div>
      
</sf-nav> 3.3 Setup Parameter Passing To enable parameter pass-down during routing, include the following Javascript code block in the sf-nav block. This code block: Listens to the routeChange event from SfNav and updates the routeMap associative array with the new route and its parameters, every time the routeChange event is fired. Exposes two functions - (1) getCurrentPathname and (2) getCurrentArgs, which can be used to obtain the current location and the current parameters. <body>
    <sf-nav>

        <h2 slot="brandName"><a href="#home" >Superflows</a></h2>
        
        <a slot="brandImage" href="#home" ><img alt="logo" src="https://superflows-images.s3.ap-south-1.amazonaws.com/superflows_black_transparent_200.png" /></a>
        
        <div slot="content">
        </div>
    </sf-nav>

    <script>
        var routeMap = [];
  
        function getCurrentPathName() {
          return window.location.hash.length === 0 ? '' : window.location.hash.split("/")[0].replace('#', '');
        }
        function getCurrentArgs() {
          return routeMap[getCurrentPathName() + '.html'];
        }
  
        document.getElementsByTagName('sf-nav')[0].addEventListener('routeChange', (e) => {
          routeMap[e.detail.pathName] = e.detail.args;
        })
    </script>

</body> 3.4 Confirm That Routing Works At this point, the routing configuration is complete. 💾 ↺ Save dist/index.html, and reload the browser. You will observe the following: The header and the Footer are now separated with a content block in between. SfNav now throws the 'Could not find home.html' message. Don't worry, this is perfectly fine because we haven't yet created the home.html file. 3.5 Create dist/home.html Create a new file dist/home.html with the below HTML: <div class="cContainer">
    <h1>Home Page</h1>
    <p>Lorem Ipsum is simply dummy text of the printing and typesetting industry. Lorem Ipsum has been the industry's standard dummy text ever since the 1500s, when an unknown printer took a galley of type and scrambled it to make a type specimen book. It has survived not only five centuries, but also the leap into electronic typesetting, remaining essentially unchanged. It was popularised in the 1960s with the release of Letraset sheets containing Lorem Ipsum passages, and more recently with desktop publishing software like Aldus PageMaker including versions of Lorem Ipsum.</p>
</div> 💾 ↺ Save dist/home.html, and reload the browser. The home page content will now load. You will observe that the content loads properly. Let us style it a bit, so that it looks a bit nicer. Add the following styles in the style block of dist/index.html: <head>
    <style>
        
        
        .cContainer {
            margin: 10px;
            padding: 20px;
            border: dash 1px gray;
        }
    </style>
</head> 💾 ↺ Save dist/index.html, and reload the browser. This looks good for now. We can surmise now that routing is working normally. Let us move ahead. 🍔 Step 4 - Setup Main Menu SfNav supports two-level menus. Creating it is pretty straightforward. Create an unordered list as shown below and set it to the main menu slot. Some menu items point to internal links (links with hash) and some menu items point to external links (links with http(s)). <body>
    <sf-nav>
        
        <ul slot="mainMenu">
            <li><a href="#about">About</a></li>
            <li class="li-solutions">
                <a href="javascript:void(0);" class="a-solutions">Solutions</a>
                <ul>
                    <li><a href="#services">Services</a></li>
                    <li><a href="#products">Products</a></li>
                </ul>
            </li>
            <li>
                <a href="javascript:void(0);">Contact Us</a>
                <ul>
                    <li><a href="https://instagram.com">Instagram</a></li>
                    <li><a href="https://facebook.com">Facebook</a></li>
                    <li><a href="https://youtube.com">YouTube</a></li>
                </ul>
            </li>
        </ul>
    </sf-nav>
</body> 💾 ↺ Save dist/index.html, and reload the browser. If you click on any menu items with internal links such as services, products, you will observe that SfNav throws the file not found message. This is because we haven't yet created these files. 📝 Quick Task Go ahead and create and with any content of your choice and observe how it is rendered. services.html products.html Drop your observations in the comments section! 🔎 Step 5 - Setup Search Search functionality is available by default. Just subscribe to the searchClick event to capture the event when the user types something in the search box and presses Enter. <script>
document.getElementsByTagName('sf-nav')[0].addEventListener('searchClick', (e) => {console.log('search clicked ' + JSON.stringify(e.detail));})
</script> 🦜 Step 6 - Setup Social Media Links Social media links can be configured using an unordered list as well. Below is an example: <body>
    <sf-nav>
        
        
        <ul slot="socialMedia">
            <li><a href="https://facebook.com"><img src="https://superflows-images.s3.ap-south-1.amazonaws.com/facebook-black_round.png" /></a></li>
            <li><a href="https://twitter.com"><img src="https://superflows-images.s3.ap-south-1.amazonaws.com/twitter_black_round.png" /></a></li>
            <li><a href="https://youtube.com"><img src="https://superflows-images.s3.ap-south-1.amazonaws.com/youtube_black_round.png" /></a></li>
        </ul>

    </sf-nav>
</body> 💾 ↺ Save dist/index.html, and reload the browser. 🔔 Step 7 - Setup Notifications 7.1 HTML Configuration The notifications section has three parts - (1) Unread notifications, (2) Read notifications and (3) View-all link. Configuration is shown below: <body>
    <sf-nav>
        
        
        <ul slot="unreadNotifications">
            <li><a href="#notification/1"><h3>Sonali Joshi</h3><p>mentioned you in a comment</p><div>1 day ago</div></a></li>
            <li><a href="#notification/2"><h3>Rahim Ahmed</h3><p>reacted to your blog post</p><div>2 days ago</div></a></li>
            <li><a href="#notification/3"><h3>John Bolton</h3><p>replied to a thread that you posted in</p><div>1 month ago</div></a></li>
        </ul>
        <ul slot="readNotifications">
            <li><a href="#notification/4"><h3>Sonali Joshi</h3><p>mentioned you in a comment</p><div>1 day ago</div></a></li>
            <li><a href="#notification/5"><h3>Rahim Ahmed</h3><p>reacted to your blog post</p><div>2 days ago</div></a></li>
            <li><a href="#notification/6"><h3>John Bolton</h3><p>replied to a thread that you posted in</p><div>1 month ago</div></a></li>
        </ul>
        
    </sf-nav>
</body> 💾 ↺ Save dist/index.html, and reload the browser. If you try clicking on any notification, you will see the file not found message. This is fine because we haven't yet created dist/notification.html. Shall we do that now? 7.2 Create dist/notification.html Create a new file dist/notification.html. Copy and paste the HTML code given below into it. <div class="cContainer">
    <h1>Notification Page</h1>
    <h2></h2>
    <p>Lorem Ipsum is simply dummy text of the printing and typesetting industry. Lorem Ipsum has been the industry's standard dummy text ever since the 1500s, when an unknown printer took a galley of type and scrambled it to make a type specimen book. It has survived not only five centuries, but also the leap into electronic typesetting, remaining essentially unchanged. It was popularised in the 1960s with the release of Letraset sheets containing Lorem Ipsum passages, and more recently with desktop publishing software like Aldus PageMaker including versions of Lorem Ipsum.</p>
</div>
<script>
    document.getElementsByClassName('cContainer')[0].children[1].innerHTML = 'Arguments: ' + JSON.stringify(getCurrentArgs())
</script> 💾 ↺ Save dist/notification.html, and reload the browser. Go ahead, and try clicking on any of the notifications. You will observe the following: The notification page will render. The URL parameters or arguments such as #notification/1 or #notification/2 will be passed down to the notification component and rendered as well. Quick Note This is essentially routing at work where the parameters from the browser address bar are fed to the notification component. © Step 8 - Copyright Notice Copyright notice can be configured in the footer as follows: <body>
    <sf-nav>
        
        
        <p slot="copyright">Copyright 2022 Superflows</p>

    </sf-nav>
</body> 💾 ↺ Save dist/notification.html, and reload the browser. The copyright notice will be displayed in the footer. 🕺 Step 9 - Setup Call To Action Button Most navbars have a call to action button. In SfNav, it can be configured as follows: <a slot="cta" href="#login">Sign In</a> 💾 ↺ Save dist/notification.html, and reload the browser. The call to action button will be displayed in the header. 👨👩 Step 10 - Setup User Profile The user profile section contains two parts - (1) The profile picture toggle and (2) The profile menu. Both parts can be configured as follows: <body>
    <sf-nav>
        
        
        <img alt="profile" slot="profilePicture" src="https://preview.keenthemes.com/metronic-v4/theme/assets/pages/media/profile/profile_user.jpg" />

        
        <ul slot="profileMenu">
            <li><a href="#settings">My Profile</a></li>
            <li>
                <a href="javascript:void(0);">Settings</a>
                <ul>
                    <li><a href="#settings_data">Data & Privacy</a></li>
                    <li><a href="#settings_notifications">Notifications</a></li>
                </ul>
            </li>
            <li>
                <a href="javascript:void(0);">Support</a>
                <ul>
                    <li><a href="#ticket">Create Ticket</a></li>
                    <li><a href="#chat">Chat With Us</a></li>
                </ul>
            </li>
            <li><a href="#signout">Sign Out</a></li>
        </ul>

    </sf-nav>
</body> 💾 ↺ Save dist/notification.html, and reload the browser. 🔴🟢🔵 Step 11 - Theme The User Interface Let us now modify the colours, to make the UI more attractive and customizable to your needs. For the sake of this tutorial, we have used some colours, but you are free to choose any colours. Add the below-given styles to the style block of dist/index.html. <style>

    sf-nav {
        --nav-background-color: rgb(4, 135, 205);
        --nav-color: #fff;
        --notif-background-color: rgb(4, 135, 205);
        --notif-color: #fff;
        --menu-background-color: rgb(49, 161, 222);
        --menu-color: #fff;
        --footer-background-color: rgb(167, 147, 60);
        --footer-color: #fff;
    }

</style> 💾 ↺ Save dist/notification.html, and reload the browser. Conclusion That is it then! I have shown you how to create a single-page web application with a modular code architecture using SfNav, a web component provided by Superflows. If you have any questions, do not hesitate to drop a comment below and I would be happy to answer. Please find a video of the same tutorial below: https://www.youtube.com/watch?v=ZGDhUaleN84&embedable=true Resources Documentation Read Demo Project View Demo YouTube Channel View Channel Join The Community Join Now Also published . here

Creating a Single Page Modular App Architecture Using Pure HTML, CSS & Javascript

A Guide to Creating Fun User Polls and Surveys With QuickBlox in Your Flutter App

Mobile Price Classification: An Open Source Data Science Project with Dagshub

. The Art of Money Getting or, Golden Rules for Making Money  by  P. T. Barnum is part of HackerNoon’s Book Blog Post series . The table of Links for this book can be found here When a man is in the right path, he must persevere. I speak of this because there are some persons who are "born tired;" naturally lazy and possessing no self-reliance and no perseverance. But they can cultivate these qualities, as Davy Crockett said: "This thing remember, when I am dead: Be sure you are right, then go ahead." It is this go-aheaditiveness, this determination not to let the "horrors" or the "blues" take possession of you, so as to make you relax your energies in the struggle for independence, which you must cultivate. How many have almost reached the goal of their ambition, but, losing faith in themselves, have relaxed their energies, and the golden prize has been lost forever. It is, no doubt, often true, as Shakespeare says: "There is a tide in the affairs of men, Which, taken at the flood, leads on to fortune." If you hesitate, some bolder hand will stretch out before you and get the prize. Remember the proverb of Solomon: "He becometh poor that dealeth with a slack hand; but the hand of the diligent maketh rich." Perseverance is sometimes but another word for self-reliance. Many persons naturally look on the dark side of life, and borrow trouble. They are born so. Then they ask for advice, and they will be governed by one wind and blown by another, and cannot rely upon themselves. Until you can get so that you can rely upon yourself, you need not expect to succeed. I have known men, personally, who have met with pecuniary reverses, and absolutely committed suicide, because they thought they could never overcome their misfortune. But I have known others who have met more serious financial difficulties, and have bridged them over by simple perseverance, aided by a firm belief that they were doing justly, and that Providence would "overcome evil with good." You will see this illustrated in any sphere of life. Take two generals; both understand military tactics, both educated at West Point, if you please, both equally gifted; yet one, having this principle of perseverance, and the other lacking it, the former will succeed in his profession, while the latter will fail. One may hear the cry, "the enemy are coming, and they have got cannon." "Got cannon?" says the hesitating general. "Yes." "Then halt every man." He wants time to reflect; his hesitation is his ruin; the enemy passes unmolested, or overwhelms him; while on the other hand, the general of pluck, perseverance and self-reliance, goes into battle with a will, and, amid the clash of arms, the booming of cannon, the shrieks of the wounded, and the moans of the dying, you will see this man persevering, going on, cutting and slashing his way through with unwavering determination, inspiring his soldiers to deeds of fortitude, valor, and triumph. About HackerNoon Book Series: We bring you the most important technical, scientific, and insightful public domain books. *This book is part of the public domain.Barnum P. T. (1980).The Art of Money Getting
  or, Golden Rules for Making Money: Project Gutenberg. Release Date: July 30, 2009 [EBook #8581] Last Updated: February 4, 2013, from [ https://www.gutenberg.org/ebooks/8581 ]( https://www.gutenberg.org/ebooks/8581 ).*This eBook is for the use of anyone anywhere at no cost and with almost no restrictions whatsoever.  You may copy it, give it away or re-use it under the terms of the Project Gutenberg License included with this eBook or online at[ https://www.gutenberg.org](http://www.gutenberg.org/ ), located at [ https://www.gutenberg.org/policy/license.html ] (https://www.gutenberg.org/policy/license.html)

The Art of Money Getting or, Golden Rules for Making Money - Chapter 4 - Persevere

Crypto Crash? Not Really... 

DynaCLI (Dynamic CLI) is a cloud-friendly, open source library for converting pure Python functions into Linux Shell commands. This article explains how makes writing Command Line Interfaces in Python easy and efficient, using as an example a function to generate a QR code that records a person’s vaccination status. DynaCLI This is a continuation of the article , which describes how to use different Python libraries like , , , , and to build CLI applications. To understand the motivations and use cases for DynaCLI, read the . To learn the differences between DynaCLI and alternatives, refer to . How to Write User-friendly Command Line Interfaces in Python argparse Click Typer docopt Fire Medium interview DynaCLI vs. Alternatives Motivation The basic idea behind DynaCLI is to accelerate and automate the process of building CLI applications as much as possible by focusing solely on Python code. Function arguments are converted to CLI commands, and DynaCLI generates help messages from the Python function docstrings. We’ll demonstrate this approach by generating a QR code that indicates a person’s vaccination status. Sounds interesting? Let’s start exploring… A conventional CLI building process Building CLIs, in general, is a two-step process: first, write the core code and, second, develop the CLI predefined as a set of arguments, as shown below. Let’s start by looking at the code from the original, reference ). article In this snippet, they are first writing methods to implement the core code. from dataclasses import dataclass, field, asdict
import requests
import shutil
from datetime import datetime

class ValidationException(Exception):
    pass

@dataclass
class Vaccination:
    manufacturer: str
    date: str

    def __post_init__(self):
        try:
            date = datetime.strptime(self.date, "%Y-%m-%d")
        except Exception:
            raise ValidationException("Vaccination date should be in format YYYY-MM-DD.")
        if date > datetime.today():
            raise ValidationException("Vaccination date should not be a future date.")
        if self.manufacturer.lower() not in ["pfizer","moderna","astrazeneca","janssen","sinovac"]:
            raise ValidationException("Your vaccine manufacturer is not approved.")

@dataclass
class QRCode:
    name: str
    birth: str
    vaccine: list[Vaccination] = field(default_factory=list)

def generate_qr_code(qr_code):
    code = asdict(qr_code)
    res = requests.get(f"http://api.qrserver.com/v1/create-qr-code/?data={code}", stream=True)
    if res.status_code == 200:
        with open("qr_code.png", "wb") as f:
            res.raw.decode_content = True
            shutil.copyfileobj(res.raw, f)
        return "QR code has been generated."
    else:
        raise Exception("QR code cannot be generated by QR Code Generator API.") Following that is the code needed to build the CLI. This example uses the argparse library (code snippet from the referenced article): def main():
    parser = argparse.ArgumentParser(description="Generate your vaccination QR code.")
    parser.add_argument("-n", "--name", type=str, help="Your name", required=True)
    parser.add_argument("-b", "--birth", type=str, help="Your birthday in YYYY-MM-DD format", required=True)
    parser.add_argument("-m", "--manufacturer", type=str, nargs="+", help="The vaccine manufacturer", required=True, choices=[
            "pfizer","moderna","astrazeneca","janssen","sinovac"])
    parser.add_argument("-d", "--date", type=str, nargs="+", help="The date of vaccination", required=True)
    args = parser.parse_args()
    
    if len(args.manufacturer) != len(args.date):
        logging.error(
            "The number of vaccine manufacturer doesn't match with the number of vaccine dates."
        )
        exit(1)
    
    qr_code = QRCode(name=args.name, birth=args.birth, vaccine=[Vaccination(args.manufacturer[i], args.date[i]) for i in range(len(args.date))])
    generate_qr_code(qr_code)

if __name__ == "__main__":
    try:
        main()
    except ValidationException as e:
        logging.error(e)
    except Exception as e:
        logging.exception(e) CLI Designing with DynaCLI With DynaCLI, we can skip the second part by designing our functions to be CLI-friendly. Functionally, the core logic is the same. To demonstrate the differences, we will update the original code as shown below. Before that, just quickly install DynaCLI to get ready: pip3 install dynacli Restructuring First of all, we would like to restructure the code. Thinking about the CLI design, there should be then feature-set (the actual Python package), which is for storing all commands, followed by to output the actual QR codes: ./qr-code green-badge generate $ tree green_badge -I __pycache__ green_badge
├── generate.py
└── __init__.py From the original code, we know that the vaccine manufacturers are a limited set of companies; this kind of information is a good fit for type Enum. from enum import Enum

class Manufacturer(Enum):
    pfizer = 1
    moderna = 2
    astrazeneca = 3
    janssen = 4
    sinovac = 5 Alternatively, we can use . I am going to use Enum here instead of Literal. By choosing this approach, we add some defensive control on input data as Enum will automatically validate the data without any redundant custom validator. Literal Therefore, we do not need a manufacturer validation inside anymore and can remove it. Going further, the whole purpose of here is validation, so we can replace this with . __post_init__ @dataclass TypedDict Our updated code file looks like this: generate.py class Vaccination(TypedDict):
    date: date
    manufacturer: Manufacturer


class QRCode(TypedDict):
    name: str
    birth: str
    vaccinations: list[Vaccination] CLI Building The second big step is to design the actual function in a way that it is going to accept all the necessary information as arguments: generate def generate(first_name: str, last_name: str, birth_date: str,  **vaccinations: Manufacturer) -> None:
    """
    Generate your vaccination QR code.

    Args:
        first_name (str): name of the vaccinated person
        last_name (str): surname of the vaccindate person
        birth_date (str): birthday of the vaccinated person in YYYY-MM-DD format
        **vaccinations (Manufacturer): vaccination information as date=manufacturer 

    Return: None
    """
    # TODO: handle duplicate or wrong dates.
    qr_code = QRCode(
        name=f"{first_name} {last_name}",
        birth=birth_date,
        vaccinations=[
            Vaccination(manufacturer=manufacturer, date=datetime.strptime(date, "%Y-%m-%d"))
            for date, manufacturer in vaccinations.items()
        ],
    )
    res = requests.get(
        f"http://api.qrserver.com/v1/create-qr-code/?data={qr_code}", stream=True
    )
    if res.status_code != 200:
        raise Exception("QR code cannot be generated by QR Code Generator API.")
    with open("qr_code.png", "wb") as f:
        res.raw.decode_content = True
        shutil.copyfileobj(res.raw, f)
    print("QR code has been generated.") Ideally, it should print a nice error message if one of the vaccination pairs contains wrong or duplicate date — but for the sake of simplicity, we just skip this step. The major change is that we are going to accept the date and the manufacturer name as key-value pairs. This is quite intuitive, isn’t it? The CLI call will look something like this: ./qr-code green-badge generate John Doe 1989-10-24 2021-01-01=pfizer 2021-06-01=pfizer The next important difference is that DynaCLI populates help messages from the docstrings in the methods containing the explanation of the arguments. …You are right, this is quite Pythonic; if the explanations are already added once, why not use the same information to build the help messages in the CLI. The rest of the code is the same — functionally the code logic is unchanged. CLI Entrypoint Now as the last step, we create the CLI entry point with DynaCLI. We have already provided the bootstrapper script, all you need is to provide the path for command: dynacli $ dynacli init qr-code path=. Successfully created CLI entrypoint qr-code at /home/ssm-user/OSS/medium-articles/how_to_convert_python_functions/code It will fill the qr-code script with some boilerplate, starter code, you need just change commented portions, in order to have a customized version of your CLI: #!/usr/bin/env python3

"""
DynaCLI bootstrap script # Change me
"""


import os
import sys
from typing import Final

from dynacli import main

cwd = os.path.dirname(os.path.realpath(__file__))

__version__: Final[str] = "0.0.0" # Change me to define your own version


search_path = [cwd] # Change me if you have different path; you can add multiple search pathes
sys.path.extend(search_path)
# root_packages = ['cli.dev', 'cli.admin'] # Change me if you have predefined root package name
# main(search_path, root_packages) # Uncomment if you have root_packages defined

main(search_path) Change the commented sections, remove redundant comments and you are ready to go: #!/usr/bin/env python3

"""
Sample QR Generator
"""

import os
import sys

from dynacli import main

cwd = os.path.dirname(os.path.realpath(__file__))

__version__ = '1.0'


search_path = [cwd]
sys.path.extend(search_path)

main(search_path) As you must have already noticed, there is no CLI pre-processing, adding arguments, version callback, etc. Everything is dead simple Python. DynaCLI grabs the version from and the CLI name from the `qr_code` docstring. version That’s it — the changes are done and we are ready to run the CLI. Getting help output: $ ./qr-code -h

usage: qr-code [-h] [-v] {green-badge} ...

Sample QR Generator

positional arguments:
  {green-badge}
    green-badge  Generate Green Badge

optional arguments:
  -h, --help     show this help message and exit
  -v, --version  show program's version number and exit The help message comes from the package file: green-badge __init__.py $ cat green_badge/__init__.py """
Generate Green Badge
""" __version__ = "2.0" Getting the version of the CLI itself: $ ./qr-code --version qr-code - v1.0 Now, let’s think about a different situation when you are porting an already developed package to be exposed by the CLI, and it has its own version set to v2.0. Should it mess with the CLI version? It is not ideal to have a single version for the package and the CLI. With DynaCLI, you can version your packages and even modules. Again, it is quite Pythonic: just add to package and to the module itself. __version__ __init__.py generate.py $ ./qr-code green-badge --version qr-code green-badge - v2.0 $ ./qr-code green-badge generate --version qr-code green-badge generate - v3.0 DynaCLI detects and the function in it. Only public names are exposed by the CLI and the function docstring is used to register the help message. generate.py generate Here, we would like to stress that, with DynaCLI, there is no need to begin writing things from scratch and redo the whole code. All you need is to import already existing functionality to an intermediate representation as we did in and register it in the CLI. generate.py This effectively conforms to the , where your original code is closed to modification but is open to being extended via CLI. Open/Closed Principle Our final version of looks like: generate.py import shutil
import requests

from datetime import date, datetime
from typing import Literal, TypedDict
from enum import Enum


__version__ = "3.0"


class Manufacturer(Enum):
    pfizer = 1
    moderna = 2
    astrazeneca = 3
    janssen = 4
    sinovac = 5


class Vaccination(TypedDict):
    date: date
    manufacturer: Manufacturer


class QRCode(TypedDict):
    name: str
    birth: str
    vaccinations: list[Vaccination]


def generate(first_name: str, last_name: str, birth_date: str,  **vaccinations: Manufacturer) -> None:
    """
    Generate your vaccination QR code.

    Args:
        first_name (str): name of the vaccinated person
        last_name (str): surname of the vaccindate person
        birth_date (str): birthday of the vaccinated person in YYYY-MM-DD format
        **vaccinations (Manufacturer): vaccination information as date=manufacturer 

    Return: None
    """
    # TODO: handle duplicate or wrong dates.
    qr_code = QRCode(
        name=f"{first_name} {last_name}",
        birth=birth_date,
        vaccinations=[
            Vaccination(manufacturer=manufacturer, date=datetime.strptime(date, "%Y-%m-%d"))
            for date, manufacturer in vaccinations.items()
        ],
    )
    res = requests.get(
        f"http://api.qrserver.com/v1/create-qr-code/?data={qr_code}", stream=True
    )
    if res.status_code != 200:
        raise Exception("QR code cannot be generated by QR Code Generator API.")
    with open("qr_code.png", "wb") as f:
        res.raw.decode_content = True
        shutil.copyfileobj(res.raw, f)
    print("QR code has been generated.") You can get help about the command(it is our Python function in fact) using: $ ./qr-code green-badge generate -h

usage: qr-code green-badge generate [-h] [-v] first_name last_name birth_date [vaccinations <name>=<value> ...]

positional arguments:
  first_name            name of the vaccinated person
  last_name             surname of the vaccindate person
  birth_date            birthday of the vaccinated person in YYYY-MM-DD format
  vaccinations <name>=<value>
                        vaccination information as date=manufacturer ['pfizer', 'moderna', 'astrazeneca', 'janssen', 'sinovac']

optional arguments:
  -h, --help            show this help message and exit
  -v, --version         show program's version number and exit In summary, the in this sample included: highlights of using DynaCLI We did not add or register any help messages — they were grabbed from the function docstrings. DynaCLI detects and registers them as pair. **kwargs <name>=<value> No special CLI pre-processing, adding arguments, or version callbacks were required. Now, it is time to run the command and generate a QR code with the arguments provided: $ ./qr-code green-badge generate Shako Rzayev 1989-10-24 2021-01-01=pfizer 2021-06-01=pfizer

QR code has been generated. That’s it. We have focused only on the core functionality, simplified basic features of a CLI application, and reshaped it to be more user-friendly. That’s how CLIs should be! The source code: how_to_convert_python_functions/code DynaCLI is an open source offering from BST LABS. Our goal is to make it easier for organizations to realize the full potential of cloud computing through a range of open source and commercial offerings. We are best known for , the Cloud AI Operating System, a development platform featuring Infrastructure-from-Code technology. BST LABS is a software engineering unit of . CAIOS BlackSwan Technologies

How to Build a User-Friendly CLI from Pure Python Functions

The first version of by in . ARTH was launched MahaDAO January 2021 What is MahaDAO? MahaDAO is a community-focused decentralized organization launching the world's first value coin, ARTH, dedicated to solving inflation problems and bringing stability to the economy. The Protocol employs a two-token system. The first being, ARTH a new type of currency designed to not be pegged to government-owned currencies (like US Dollar, Euro, or Chinese Yuan), but still remain relatively stable (unlike Gold and Bitcoin). Without being influenced by government-owned currencies, ARTH is backed by GMU(Global measurement Unit) and the Second token is governance token MAHA. Overview Since the team launched the first version of ARTH in January 2021 the team tried and tested different types of models to build a stable value coin until June 2021, whether it is a fully collateral-backed model, elastic supply model, Seigniorage Share model, or partial collateral model. The team tested all and finally got stability. It was founded after which we can scale it through different products. here I remember a line by Paul Graham "In order to scale, you have to first do things that don't scale at all". In this article, I will explain my experiences and journey of building a platform to scale the ARTH. "In order to scale, you have to first do things that don't scale at all" What's next? After getting stability, the core strategy was to build a platform that can scale the ARTH ecosystem for larger adoption. That's where the ARTH Loan came into the picture. A product like this had always been a part of MahaDAO's long-term plan. A product where users can take the most advantage through value coin without worrying about volatility so building ARTH Loan from which users can take a loan with zero interest and generate ARTH with leveraged position would be the right and perfect opportunity to grow market cap and users. Product Vision I was the sole designer to work on an end-to-end process to design a cross-platform experience, a design system that allows us not only to design and build things more efficiently but also helps evolve the product faster multiple times a year. My Role and Responsibilities - 0% interest loan platform that will use ARTH value coin as its primary asset to transact loans. Borrowers can take leverage and extract loans by collateralizing 110% of their crypto assets. - Users can redeem ARTH using different collaterals. - Users can put different LP tokens to earn reward MAHA & ARTH - It's a one-stop place to analyze protocol. apart from token stats, there are graphs to analyze ARTH loan stats, Revenue, and ARTH utilization. Use-cases that we were focusing on after getting ARTH stability: 1. ARTH Loan 2. Redeem 3. Farming 4. Analytics What is the end goal? This project aimed to ship the entire product experience on the web and mobile. While testing ARTH for stability and different launches earlier, we found out that a decent number of users were coming from mobile devices also. So, the product should be fully responsive and consistent across all the devices. As the ecosystem was growing, it was clear that major product design changes needed to happen to give simplicity. ARTH's design needs to scale. The tricky constraint here was to move fast to build since we decided to launch ARTH Loans within 5 weeks after coming up with the idea. Designing complex product can be difficult but by dividing it into important user experiences, you can make it approachable and double. Background In January, I worked on the Vault app which was the first version for ARTH adoption. we aimed to scale the adoption of ARTH through Vaults. In Vaults, people can put collateral and generate ARTH. Their locked collateral will also generate some interest in the vault. but we stopped working on this product without launch and focused on testing value coins to stabilize ARTH with different models. Approach and behind the process Product vision was set, now it was time for action. From early launches of testing ARTH for stability and testing with the public community and tester groups, our intention for the next product was to have a consistent experience across web and mobile devices. The process began by sketching out the user flows, considering all the design use cases, and progressed to iterating on Figma prototypes. I explored a lot of different flows and design use cases, after few rounds of iteration and feedback from a team I had a solid base of user flows to work from. Below, I have put images of a few user flows, early research, and visual iterations to show the process. Apart from these, there were lots of papers which I haven't put here. Early iterations and flows A glimpse of the early version of ARTH before getting stability Early Exploration As I was the first designer of MahaDAO, from the start I considered working on building a scalable design system from the ground up which can be useful to build products as it scales and keep the entire product ecosystem consistent on a multitude of platforms and devices even with multiple introductions of new features. It truly helped to reduce the time to build and design multiple products by using existing design patterns. For the building design system, I approached the Atomic Design methodology. as the different products grew at a scale I added more patterns to make evolving system. I added these patterns to create a component library which I referred to throughout the design process. Making a whole design system from scratch was challenging considering time constraints but while iterating on designs it helped to increase productivity and I was able to focus more on solving design problems to experiment with more ideas faster. "A Design System isn't a Project. It's a Product, Serving Products. " - Nathan Curtis Rethinking primary navigation As a part of product design scaling, navigation was a major redesign. In a previous version of ARTH, I went with the top navigation and it worked well when there were not too many navigation items. but as our product became more sophisticated I have to rethink navigation and after a few iteration and testing, I decided to go with side navigation to simplify the experience. By using side navigation I was able to show at least double the number of navigation links. This also means it is much easier to add more items as your product grows and keep the navigation view consistent across all the screen sizes. Overview and new loan The overview page is where users can see their loan and protocol loans and major data at glance, there are 3 sections to see what's happening with loan protocol at glance. There are two major data points that need to be displayed on the overview page: Total locked value in protocol and total ARTH debt of protocol. Users should be able to see their loan information at glance instead of opening individual tabs to see information and the third section is the all loan section where users can see all loans of users from protocol. In the third section, if there is any user whose loan's collateral ratio comes down to below 110% then any user can liquidate that loan. I starting there are four collateral available like ETH, MATIC, DAI, WBTC from which users can open loans per collateral. while opening the loan they can also select leverage to open a loan with the leveraged position. Modify Loan Modify loans was quite complicated where there are different functions like add or remove collateral from loans or pay part of the debt and generate more ARTH for loans. so I created two tabs for updating collateral or tabs for categorizing selection. Close Loan With enough ARTH, users can close their loan and if users don't have a collaterals ratio above 110% then the user's loan will be liquidated and collateral will be distributed to Stability Pool as a reward to Stability Providers. Stability Pool As a secondary measure to avoid the protocol going into extreme stress, the stability pool acts as a source of liquidity to repay debt from liquidated loans. It ensures that the total ARTH supply always remains backed by collateral. When a loan position is liquidated, an equivalent amount of ARTH corresponding to the remaining debt has to be burned from the Stability Pool's balance to repay its debt. In exchange, the entire collateral (committed by the borrower) is transferred to the Stability Pool as a reward to Stability Providers. The Stability Pool is funded by users transferring their ARTH into the pool. These are also known as pooled users or stability providers. Redeem From redeem, screen users can redeem ARTH by using different collaterals. Protocol Analytics The analytics page is a one-stop place to analyze all data of protocol. apart from token stats, there are graphs to analyze ARTH loan stats, Revenue, and ARTH utilization. Farming Farming is where users come to put their LP tokens from different third-party platforms and earn rewards. My goal was to make it simple so that users can see their locked amount and reward on the same screen and do all action also, instead of showing it on a different screen for a particular farming pool. Fees generated through the protocol in various ways are shared with stakers participating in these farming pools. Here I have only discussed ARTH Loan, Redeem, Analytics and Farming. Apart from these, there were other design use cases also like transaction feedback, transaction notifications, Connect wallet, user wallet which I haven't discuss here since they are easily understandable. Cross-platform experience As I discussed before, the Platform needs to have a consistent experience across different platforms and we can't compromise with user experience. so I keep that in mind and made it possible. below I have put a few of the screens to show responsive design. The Path Forward ARTH Loan is just beginning for ARTH adoption. MahaDAO will launch a Lending and borrowing platform soon which I already started to work on. Secondly, to grow the ARTH ecosystem even further MahaDAO incubated Scallop which is the world's first DeFi enabled neo-bank that uses blockchain to create a secure and safe bridge between traditional and decentralized finance. It provides a wide range of traditional banking services powered by DeFi and blockchain technology. The scallop is expected to launch its full suite of products in September 2021 in the EU and UK for 747m potential users and expand to Asia next year. Learning from building ARTH Experience of building products in the crypto space was something new and challenging to me as it's still evolving space and we are seeing new innovations every day. While doing these I faced a steep learning curve, I continuously tried and dive deep into different crypto products to learn about DeFi space more every day. From the product side, there were times where I have to change a whole product from scratch because the protocol that we tried was not working as we intended. but through this experience, the biggest learning was to building, test, and scaling ideas in the blockchain space. Sometimes, users do not have the time and patience for you to come through, which is why it's crucial to never stop learning, building, and shipping. Always make sure you deliver what you promised first and then move forward to make it the best. On an ending note, I would say building products in the crypto space can be crazy because crypto space moves too fast so you have to keep up with the pace. It's a perfect time to be working on decentralized finance and blockchain, and I'm humbled to work on design efforts as we enter the next era of Web 3.0. Learning to find your own process and thinking is challenging but if you got somebody with great experience and mindset it really helps you. Thanks to who helped whenever needed throughout the journey to make this process existing while building all the products to make important product decisions and also ingrain a habit in me to observe ask as many as "whys" for every decision whether it's from a design, product, or business perspective. I really appreciate his support to bring the best out of me. I tried to cover all the important details here. there is more to talk about but that's for another article. Thank you for reading this far. ♥️ Dalpat Prajapati Are you working on blockchain or Defi Project? Let's connect. I'm available on , , Twitter Linkedin Instagram This article was . first published here

How ARTH Stablecoin Was Built and Designed to Protect The User's Purchasing Power

#HackerNoon Reporter: Please tell us briefly about your background. I’m Mitja Goroshevsky, the Co-Founder and CTO of TON Labs, where I focus on developing core technical infrastructure for , a community-driven blockchain underpinning an open and free internet. Free TON Before Co-Founding TON Labs in 2018, I spent over two decades in the IT industry, establishing various successful businesses such as , the first international voice over IP provider; Internet Telecom, a telecoms and enterprise-grade IP solution provider; and BrainsMatch, a Machine Learning company. Following a productive stint in the telecoms and IT space, I ventured into the world of blockchain and served as the Chief Architect for US-based . DeltaThree Communications ShelterZoom Aside from my expertise in entrepreneurship, IT and blockchain, I also participate in various research projects and have co-authored several patents in distributed computing. #What's your startup called? And in a sentence or two, what does it do? is building Web Free: a meritocratic decentralized web, focused on scalable blockchain infrastructure for global adoption. TON Labs We’re a company composed of experts in decentralization, including leading developers and creative technologists. As the core developer of , a blockchain that enables anyone to create and manage decentralized organizations, TON Labs is committed to optimizing the development of applications easily accessible to global communities and enterprises. Free TON #What is the origin story? Oh, it’s an interesting story - the founders of TON Labs all knew each other beforehand - but we met when creating Industry 4.0 - a venture fund where blockchain was high on the mandate. There was serious research done on the industry. When the TON project became public and the white paper available it became clear what transformational potential this technology had. The team decided to create TON Labs with a mission to develop a comprehensive operating system on top of TON.  Later on the team earned trust of the community and became core developer of TON, merging it with TON OS into a powerful vertical technology stack - Free TON. #What do you love about your team, and why are you the ones to solve this problem? I can honestly say I feel we have the greatest team ever. Everyone is super motivated, dedicated and extremely talented. We would not be able to achieve what we do without such a team. We are all remote and distributed globally, but all gather together twice a year for offsites in exciting destinations and get along like family. #If you weren’t building your startup, what would you be doing? Building another startup, creating or learning. #At the moment, how do you measure success? What are your core metrics? By doing what others cannot do. It’s not about the numbers for me. Transaction speeds and token price do not equate success, rather creating a sustainable network, underpinned by endurance and reliability - one that is on-demand scalable and ready to meet any needs is what matters. Our community contributes to the network for rewards and the technology we have implemented allows for this trustless economy, for limitless DAOs (or subgovernances as well call it) to realise visions and contribute to an open, free internet. The fact that we have never had an ICO nor will ever have one combined with our core ethics of decentralization speaks volumes. Free TON will never go corporate. We will never ‘sell out’ and become centralized and that is what I call success. #What’s most exciting about your traction to date? The last lap in the Rust Cup contest resulted in some very exciting news. We officially broke world record in TPS on a stable network. Following this contest, the Rust testnet will be launched, afterwards the migration to mainnet will take place. This is when we will go full force on demonopolizing and decentralizing the internet. #What technologies are you currently most excited about, and most worried about? And why? Honestly, I am most excited about our own. Just have a look at our Whitepaper #What drew you to get published on HackerNoon? What do you like most about our platform? Hackernoon is a great resource for information and interesting opinions and alternative views. It would be great to partner and work on decentralizing HackerNoon! /Smirk. #What advice would you give to the 21-year-old version of yourself? Get prepared for answering questions like this from reporters! (I am not one for the spotlight--my desires lie in supporting my team and building tech.) #What is something surprising you've learned this year that your contemporaries would benefit from knowing? Everyone needs at least one pair of funky shoes.

Startup Interview With Mitja Goroshevsky - Co-Founder and CTO of TON Labs

This is the second and final part of the series on How to build and deploy an NLP model with FastAPI. In , we looked at how to build an NLP model that can classify movie reviews into different sentiments. the first part In this second and final part, you will learn What is FastAPI and how to install it. How to deploy your model with FastAPI. How to use your deployed NLP model in any Python application. So let’s get started.🚀 What is FastAPI? FastAPI is the fast and modern python web framework for building different . It provides higher performance, easier to code, and comes up with automatic & interactive documentation. FastAPI is built upon two major python libraries – (for web handling) and (for data handling & validation). FastAPI is very fast compared to Flask because it brings asynchronous function handlers to the table. APIs Starlette Pydantic If you want to know more about FastAPI, I recommend you read this by Sebastián Ramírez. article In this article, we will try to use some of the features presented in FastAPI to serve our NLP model. How To Install FastAPI Firstly, make sure you install the latest version (with pip): pip install fastapi You will also need an ASGI server for production such as . uvicorn pip install uvicorn Deploy NLP Model with FastAPI In this section, we are going to deploy our trained model as a REST API with FastAPI. The code for our API will be saved in a python file called , this file will be responsible for running our FastAPI app. NLP main.py Import packages The first step is to import packages that will help us to build the FastAPI app and run the NLP model. string punctuation nltk.tokenize word_tokenize nltk nltk.corpus stopwords nltk.stem WordNetLemmatizer re os os.path dirname, join, realpath joblib uvicorn fastapi FastAPI # text preprocessing modules from import # text preprocessing modules from import import from import from import import # regular expression import from import import import from import Initializing a FastAPI App Instance We can use the following code to initialize the FastAPI app. app = FastAPI(
    title= ,
    description= ,
    version= ,
) "Sentiment Model API" "A simple API that use NLP model to predict the sentiment of the movie's reviews" "0.1" As you can see we have customized the configuration of our FastAPI application by including: Title of the API Description of the API. The version of the API. Load the NLP model To load the model we use method and add the path to the model directory. The name of the  NLP model is . joblib.load() sentiment_model_pipeline.pkl open(
    join(dirname(realpath(__file__)), ), ) f:
    model = joblib.load(f) # load the sentiment model with "models/sentiment_model_pipeline.pkl" "rb" as Define a Function to Clean the Data We will use the same function called from Part 1 that cleans the review data by removing stopwords,  numbers, and punctuation, and finally, convert each word into its base form by using the lemmatization process in the NLTK package. text_cleaning() text = re.sub( , , text)
    text = re.sub( , , text)
    text = re.sub( , , text)
    text = re.sub( , , text) text = .join([c c text c punctuation]) remove_stop_words: stop_words = stopwords.words( )
        text = text.split()
        text = [w w text w stop_words]
        text = .join(text) lemmatize_words:
        text = text.split()
        lemmatizer = WordNetLemmatizer()
        lemmatized_words = [lemmatizer.lemmatize(word) word text]
        text = .join(lemmatized_words) text : def text_cleaning (text, remove_stop_words=True, lemmatize_words=True) # Clean the text, with the option to remove stop_words and to lemmatize word # Clean the text r"[^A-Za-z0-9]" " " r"\'s" " " r"http\S+" " link " r"\b\d+(?:\.\d+)?\s+" "" # remove numbers # Remove punctuation from text "" for in if not in # Optionally, remove stop words if # load stopwords "english" for in if not in " " # Optionally, shorten words to their stems if for in " " # Return a list of words return Create Prediction Endpoint The next step is to add our prediction endpoint called " " with the GET request method. /predict-review @app.get("/predict-review") “An API endpoint is the point of entry in a communication channel when two systems are interacting. It refers to touchpoints of the communication between an API and a server.” Then we define a prediction function for this endpoint. The name of the function is called with a review parameter. predict_sentiment() The predict_sentiment() function will do the following tasks. Receive the movie review. Clean the movie review by using the function. text_cleaning() Make a prediction by using our NLP model. Save the prediction result in the variable (either 0 or 1). output Save the probability of the prediction in the variable and format it into 2 decimal places. probas Finally, return prediction and probability result. cleaned_review = text_cleaning(review) prediction = model.predict([cleaned_review])
    output = int(prediction[ ])
    probas = model.predict_proba([cleaned_review])
    output_probability = .format(float(probas[:, output])) sentiments = { : , : } result = { : sentiments[output], : output_probability} result @app.get("/predict-review") : def predict_sentiment (review: str) """
    A simple function that receive a review content and predict the sentiment of the content.
    :param review:
    :return: prediction, probabilities
    """ # clean the review # perform prediction 0 "{:.2f}" # output dictionary 0 "Negative" 1 "Positive" # show results "prediction" "Probability" return Here are all blocks of codes in the file. main.py string punctuation nltk.tokenize word_tokenize nltk nltk.corpus stopwords nltk.stem WordNetLemmatizer re os os.path dirname, join, realpath joblib uvicorn fastapi FastAPI

app = FastAPI(
    title= ,
    description= ,
    version= ,
) open(
    join(dirname(realpath(__file__)), ), ) f:
    model = joblib.load(f) text = re.sub( , , text)
    text = re.sub( , , text)
    text = re.sub( , , text)
    text = re.sub( , , text) text = .join([c c text c punctuation]) remove_stop_words: stop_words = stopwords.words( )
        text = text.split()
        text = [w w text w stop_words]
        text = .join(text) lemmatize_words:
        text = text.split()
        lemmatizer = WordNetLemmatizer()
        lemmatized_words = [lemmatizer.lemmatize(word) word text]
        text = .join(lemmatized_words) text cleaned_review = text_cleaning(review) prediction = model.predict([cleaned_review])
    output = int(prediction[ ])
    probas = model.predict_proba([cleaned_review])
    output_probability = .format(float(probas[:, output])) sentiments = { : , : } result = { : sentiments[output], : output_probability} result # text preprocessing modules from import # text preprocessing modules from import import from import from import import # regular expression import from import import import from import "Sentiment Model API" "A simple API that use NLP model to predict the sentiment of the movie's reviews" "0.1" # load the sentiment model with "models/sentiment_model_pipeline.pkl" "rb" as # cleaning the data : def text_cleaning (text, remove_stop_words=True, lemmatize_words=True) # Clean the text, with the option to remove stop_words and to lemmatize word # Clean the text r"[^A-Za-z0-9]" " " r"\'s" " " r"http\S+" " link " r"\b\d+(?:\.\d+)?\s+" "" # remove numbers # Remove punctuation from text "" for in if not in # Optionally, remove stop words if # load stopwords "english" for in if not in " " # Optionally, shorten words to their stems if for in " " # Return a list of words return @app.get("/predict-review") : def predict_sentiment (review: str) """
    A simple function that receive a review content and predict the sentiment of the content.
    :param review:
    :return: prediction, probabilities
    """ # clean the review # perform prediction 0 "{:.2f}" # output dictionary 0 "Negative" 1 "Positive" # show results "prediction" "Probability" return Run the API The following command will help us to run the FastAPI app we have created. uvicorn main:app --reload Here are the settings we have defined for uvicorn to run our FastAPI app. the file main.py that has the FastAPI app. main: the object created inside of main.py with the line app = FastAPI(). app: : Enables the server to automatically restart whenever we make changes in the code. --reload FastAPI provides an Automatic Interactive API documentation page. To access it navigate to in your browser and then you will see the documentation page created automatically by FastAPI. http://127.0.0.1:8000/docs The documentation page shows the name of our API, the description, and its version. It also shows a list of available routes in our API that you can interact with. To make a prediction first click the " " route and then click on the button " , it allows you to fill the review parameter and directly interact with the API. predict-review Try it out" Fill the review field by adding a movie review of your choice. I added the following movie review about movie released in 2021. Zack Snyder's Justice League "I loved the movie from the beginning to the end. Just like Ray fisher said, I was hoping that the movie doesn't end. The begging scene was mind blowing, liked that scene very much. Unlike 'the Justice League' the movie show every hero is best at their own thing, make us love every character. Thanks, Zack and the whole team." Then click the execute button to make a prediction and get the result. Finally, the result from the API shows that our NLP model predicts the review provided has sentiment with the probability of . Positive 0.70 Use NLP model in any Python Applications To use our NLP API in any python application, we need to install the requests python package. This python package will help us to send HTTP requests to the FastAPI app we have developed. To install the requests package run the following command. pip install requests Then create a simple python file called . This file will be responsible to send our HTTP requests. python_app.py We first import requests package. requests r import as Add a movie review about Movie. Godzilla vs Kong (2021) review = # add review "This movie was exactly what I wanted in a Godzilla vs Kong movie. It's big loud, brash and dumb, in the best ways possible. It also has a heart in a the form of Jia (Kaylee Hottle) and a superbly expressionful Kong. The scenes of him in the hollow world are especially impactful and beautifully shot/animated. Kong really is the emotional core of the film (with Godzilla more of an indifferent force of nature), and is done so well he may even convert a few members of Team Godzilla." Then add the review in a key parameter to pass to the HTTP request. keys = { : review} "review" Finally, we send a request to our API to make a prediction of the review. prediction = r.get( , params=keys) "http://127.0.0.1:8000/predict-review/" Then we can see the prediction results. results = prediction.json()
print(results[ ])
print(results[ ]) "prediction" "Probability" This will show the prediction and its probability. Here are the results. Positive 0.54 Wrapping Up Congratulations 👏👏, you have made it to the end of this part 2. I hope you have learned something new on how to deploy your NLP model with FastAPI. If you want to learn more about FastAPI, I recommend taking this created by . full FastAPI course Bitfumes You can download the project source code  used in this article here: https://github.com/Davisy/Deploy-NLP-Model-with-FastAPI If you learned something new or enjoyed reading this article, please share it so that others can see it. Until then, see you in the next article!. You can also find me on Twitter and you can read more articles like this . @Davis_McDavid here Want to keep up to date with all the latest in machine learning? Subscribe to our newsletter in the footer below!

How To Build and Deploy an NLP Model with FastAPI: Part 2

The is a python module that provides the interface for interacting with the underlying that is running. OS module operating system Python This module provides a portable way of using dependent functionality. operating system comes by default on y modules therefore you don’t need to install anything to  start using it OS module Python’s standard utilit OS module provides a ton of of which you use in variety of situation in with the ranging  from s to and hem. methods interacting operating system creating new file & folder renaming deleting t You can view all of them by using as shown below dir() >>> os 
>>>dir(os) , , , , , , , , , , , , , , , , , , , , , , ............................and so on........................... import 'chown' 'chroot' 'close' 'closerange' 'confstr' 'confstr_names' 'cpu_count' 'ctermid' 'curdir' 'defpath' 'device_encoding' 'posix_fallocate' 'pread' 'putenv' 'pwrite' 'read' 'readlink' 'readv' 'remove' 'removedirs' 'removexattr' 'rename' 'renames' On this tutorial  We will just check some of methods on OS Module os.getcwd ( ) This method is used to get the of the current working . OS path directory >>> os 
>>>os.getcwd() #getting path current directory import of '/home/kalebu' os.chdir ( ) This method is used to the current directory into a new directory, it receives one parameter which is the of the new directory. OS change path >>> os 
>>>os.getcwd() #getting path before changing dir >>>os.chdir( ) 
>>>os.getcwd() #getting path after changing dir import '/home/kalebu' 'Desktop' '/home/kalebu/Desktop' os.chdir ( ) This method is used to the current directory into a new directory, it receives one parameter which is the of the new directory. OS change path >>> os 
>>>os.getcwd() #getting path before changing dir >>>os.chdir( ) 
>>>os.getcwd() #getting path after changing dir import '/home/kalebu' 'Desktop' '/home/kalebu/Desktop' ) os.listdir ( This method returns all and on the current directory. OS files sub-directory >>> os 
>>>all_dirs = os.listdir()
>>>print(all_dirs)
[ , , ] import 'documentation' 'app.py' 'os methods .png' Also, you can a parameter of a specific directory to view its file and sub-directory pass >>> os
>>> doc_dirs = os.listdir( )
>>> print(doc_dirs)
[ , , ] import 'documentation' 'learn.py' 'requirements.txt' 'you.py' os.mkdir ( ) This method is used to create a new directory of the current path, for instance, let’s create a named OS directory Products. >>> os 
>>>os.listdir() #before any change
[ , , ]
>>>os.mkdir( ) # os.mkdir(directory_name)
>>>os.listdir() #after making a dir
[ , , , ] import 'documentation' 'app.py' 'os methods .png' 'Products' new 'documentation' 'Products' 'app.py' 'os methods .png' os.rename ( ) This Module is used to rename a or , it receives two-argument, an and a of file or directory. For instance, let’s rename to OS file folder old name new name app.py os.py >>> os
>>>os.listdir()
[ , , , ]
>>> os.rename( , ) # os.rename( , )
>>> os.listdir()
[ , , , ] import 'documentation' 'Products' 'app.py' 'os methods .png' 'app.py' 'os.py' 'oldname' 'newname' 'documentation' 'Products' 'os.py' 'os methods .png' os.rmdir ( ) This method is used to a directory it receives an argument of the of the directory you want to remove. will only work for empty dirs OS remove name os.rmdir() for instance, Let’s remove the directory using . Products rmdir() >>> os
>>>os.listdir()
[ , , , ]
>>> os.rmdir( )
>>> os.listdir()
[ , , ] import 'documentation' 'Products' 'app.py' 'os methods .png' 'Products' 'documentation' 'os.py' 'os methods .png' os.remove ( ) This OS Method is used to remove or delete a file path . For instance let’s remove y file os.p >>> os
>>>os.listdir()
[ , , ]
>>> os.rmdir( )
>>> os.listdir()
[ , ] import 'documentation' 'os.py' 'os methods .png' 'os methods .png' 'documentation' 'os.py' os.sytem ( ) This Method is used to run a shell command on the python program as if you were in shell. For instance, let’s run a command using the system method. OS tree >>> os 
>>> os.system( )
.
├── documentation
│   ├── learn.py
│   ├── requirements.txt
│   └── you.py
└── os.py directory, files import 'tree' 1 4 0 os.uname ( ) This Method returns information identifying the current operating system. OS >>> os 
>>> os.uname()
posix.uname_result(sysname= , nodename= , release= , version= , machine= ) import 'Linux' 'kalebu-PC' '4.15.0-30deepin-generic' '#31 SMP Fri Nov 30 04:29:02 UTC 2018' 'x86_64' [] os.environ is not a method in the module rather than it’s a process parameter through which we can access the environment variables of the system. environ OS For instance Let’s access environment variable HOME >>> os
>>> os.environ[ ] import 'HOME' '/home/kalebu' We also set our own as shown below environment variable >>> os
>>> os.environ[ ] = #setting our own variable
>>> os.environ[ ] #accesing it import 'my_secret_key' "Life sucks" 'my_secret_key' 'Life sucks' os.getuid ( ) This Module returns  the current process’s user id. >>> os
>>> os.getuid() import 1000 os.getpid ( ) Returns the real process ID of the current process. >>> os
>>> os.getpid() import 19521 Summary In this Tutorial we learned about and how can you use  it in a variety of operating system related tasks using its methods. OS Module Hope you find it interesting, now don’t be shy share it with your fellow peer on Twitter( ) and other dev communities. Tweet now In case of any suggestion, comment, or any difficulty drop it in the comment box below and I will get back to you ASAP. The original article can be found on kalebujordan.com

The Basics of Python OS Module

In this article, I’ll show you how to create with Python3.7 + Serverless lambda Assets Monitor My name is Uria Franko and I’m a free-lancer developer Here is the plan Setup Serverless framework Get AWS credentials Write backend of the with python3.7 Assets Monitor Deploy backend to Lambda Schedule running Before we start In order to go through this tutorial, make sure you have: Installed Python 3.7 Installed Node.js v6.5.0 or later AWS account with admin rights (You can create one for free ) right here Step 1: Serverless framework Let’s start from the beginning. In order to easily write and deploy our lambda we will use this awesome framework called . It’s written on NodeJS so we need npm to install it. Let’s go: Serverless npm -g serverless install After this, let’s create a new project from template: serverless create --template aws-python3 --path my-assets-monitor It will create a new folder with two files: my-assets-monitor — a template for your Python code handler.py — configuration file serverless.yml Step 2: AWS Credentials This is the best part of this process, because once you’ve got credentials, you’ll never deal with AWS again. It’s super easy: Log in to your AWS Console, go to and click on “Add User” blue button. My Security Credentials > User Specify username (something like “serverless-admin”) and choose only “Programmatic access” checkbox. On the second page, choose “Attach existing policies directly” and look for “Administrator Access”. Once you created the user, copy your “Access key ID” and “Secret access key”. This is what you actually need to continue. (Tip: save them where you can find them ;) ) Congrats. You’ve got your keys. Open your terminal in and execute: serverless config credentials --provider aws --key xxxxxxxxxxxxxx --secret xxxxxxxxxxxxxx Step 3: Write Assets Monitor Code I’m not going to teach you how to write in Python, so just copy this code and paste to your `handler.py` file: json os sys concurrent.futures ThreadPoolExecutor urllib.parse urlparse

here = os.path.dirname(os.path.realpath(__file__))
sys.path.append(os.path.join(here, )) requests bs4 BeautifulSoup mailer Mailer

mailer = Mailer(os.environ[ ], os.environ[ ], os.environ[ ],)
internal_urls = set()
external_urls = set() ThreadPoolExecutor(workers) ex:
        res = ex.map(func, args) list(res) parsed = urlparse(url) bool(parsed.netloc) bool(parsed.scheme) mailer
    resp = requests.get(url) resp.status_code > :
        mailer.assets.append(url) :
        response = requests.get(url, timeout= )
        response.raise_for_status()
        soup = BeautifulSoup(response.content, ) requests.exceptions.ConnectTimeout err:
        errors = [ ]
        mailer.send_errors(errors) requests.exceptions.ConnectionError err:
        errors = [err]
        mailer.send_errors(errors) requests.exceptions.HTTPError err:
        errors = [ ]
        mailer.send_errors(errors) soup urls = set()
    url_parsed = urlparse(url)
    domain_name = url_parsed.netloc
    soup = request_url(url) soup : a_tag soup.findAll([ , , , ]):
        source = a_tag.name == a_tag.name == :
            source = href = a_tag.attrs.get(source) href == href href: parsed_href = urlparse(href) parsed_href.netloc == : href[ ] == :
                href = url_parsed.scheme + + domain_name + href :
                href = url_parsed.scheme + + domain_name + + href is_valid(href): href internal_urls: domain_name href: href external_urls:
                external_urls.add(href) urls.add(href)
        internal_urls.add(href) urls crawled_links = get_all_website_links(os.environ[ ]) crawled_links :
        response = { : , : } response
    multi_threading(check_status, crawled_links, )
    mailer.send_mail()
    response = { : ,
    } response import import import from import from import "./vendored" import from import from import 'TARGET_URL' 'SOURCE_EMAIL' 'DESTINATION_URL' : def multi_threading (func, args, workers) with as return : def is_valid (url) return and : def check_status (url) global if 399 : def request_url (url) try 5 "html.parser" except as 'Connection timed out to your target' return False except as return False except as f'Your target raised <strong> </strong> status code' {response.status_code} return False return : def get_all_website_links (url) if is False return False for in "a" "link" "img" "script" 'src' if "a" or "link" 'href' if "" or is None or '#' in continue if "" if 0 "/" "://" else "://" "/" if not continue if in continue if not in if not in continue return : def main (event, context) 'TARGET_URL' if is False "statusCode" 500 "body" "Error raised trying to get the target" return 20 "statusCode" 200 return Now create new file in the same directory and call it mailer.py, now copy this code and paste it in: boto3 os aws_access_key_id = os.environ[ ]
        aws_secret_access_key = os.environ[ ]
        self.client = boto3.client( , aws_access_key_id = aws_access_key_id,
                                   aws_secret_access_key = aws_secret_access_key,
                                   region_name = )

        self.target_email = source_email target_email :
            self.target_email = target_email
        self.base_url = base_url
        self.source_email = source_email
        self.assets = [] subject = body = link self.assets:
            body += self.send(subject, body) subject = body = err errors:
            body += self.send(subject, body) self.client.send_email(
            Source = self.source_email,
            Destination = { : [
                    self.target_email,
                ],
            },
            Message = { : { : subject, : }, : { : { : body, : }
                }
            },
            ReplyToAddresses = [
                self.source_email,
            ],
        ) import import : class Mailer : def __init__ (self, base_url, source_email, target_email = None) 'AWS_KEY' 'AWS_SECRET' 'ses' 'us-east-1' if is not None : def send_mail (self) "Assets Monitor Asset Failure" f"""
        <h2>There's a problem with one of your assets!</h2>
        <h4>Base URL: <a href= > </a></h4>
        """ {self.base_url} {self.base_url} for in f'<a href=" "><p> </p></a><br>' {link} {link} : def send_errors (self, errors) "Assets Monitor Website Failure" f"""
        <h3>There's a problem with your monitored website:</h3>
        <h4> </h4>
        """ {self.base_url} for in f"<p> </p><br>" {err} : def send (self, subject, body) 'ToAddresses' 'Subject' 'Data' 'Charset' 'UTF-8' 'Body' 'Html' 'Data' 'Charset' 'UTF-8' If you are lazy enough, just fork or clone it from my GitHub . repo Also, you will need to create requirements.txt file and write: requests
bs4 and execute this command to install it locally: pip install -r requirements -t vendored .txt Step 4: Deploy to AWS Lambda Pretty much like on Heroku, all you need is one configuration file “serverless.yml”. Go ahead and edit yours to make it look like this: service: my-asset-monitor

provider:
  name: aws
  runtime: python3 stage: dev
  region: us-east environment:
    AWS_KEY: AWS_SECRET: TARGET_URL: SOURCE_EMAIL: DESTINATION_EMAIL: functions:
  post:
    handler: handler.main
    events:
      - schedule:
          name: asset-checker-schedule
          description: rate: rate( minutes) .7 -1 "" "" "" "" "" 'Schedule asset checking every 30 minutes' 30 Notice that you need to change AWS_KEY -> Your AWS key that we created AWS_SECRET -> Your AWS key that we created TARGET_URL -> The website that you want to monitor (Ex: 'https://google.com') SOURCE_EMAIL -> Your verified email that you can get/verify here DESTINATION_URL -> The email that the notifications will be sent to You can also change the rate(30 minutes) to what ever you want... you can find the rates syntax here And the magic happens once you execute this in your terminal: serverless deploy It will pack all your files into .zip archive and upload to AWS, then it will create AWS CloudEvent to schedule and run the monitor automatically The serverless backend of your monitor is now live and running. Now if any asset or link in your website will be broken (Not found/ Server error / etc) you will get an email that's looks like that: Also if there's any error accessing your target you will get an email like that: Keep in mind At the beginning it will cost you nothing, because The AWS Lambda free usage tier includes 400,000 GB-seconds of compute time per month. But if you have any other lambda function or you multiply the monitors it can get to a point where they charge you... Our current usage is 60/30 * 60 * 24 * 30 = 86,400 calls each calls is between 1-5 seconds depends on the assets amount and a usage of 256MB So 86,400 * 3 (Average)  / 4 (GB-sec / 256MB = 3.9xxx) = 66,355 seconds of usage per month Serverless approach is very convenient in some cases because it allows you to build very scalable solution. It also change your thinking paradigm about managing and paying for servers. Serverless framework gives you very simple tool to deploy your function with no needing to know AWS or any other cloud provider. AWS gives you nice free tier period for services so you can build your MVPs totally for free, go live and start paying only if you reach a certain amount of users.

Assets Monitor as a Function

Asymbol of American prosperity was obliterated on September 11, 2001, along with more than 3,000 innocent American lives. It is a day (in the words of President Roosevelt after the attack on Pearl Harbor) that will live in infamy. 9/11 was a dark day that continues to cast a long shadow on the lives of affected Americans. It was also a dark day for privacy in America. Just 45 days after UA 175 and AA 11 hurtled into the towers, the USA PATRIOT Act hurtled through congress unabated, and virtually undebated. Just one day after its introduction on October 23, 2001, the Patriot Act passed in the house by a margin of nearly 300 votes; before moving on to 98–1 approval in the Senate on October 25th. USA PATRIOT stands for ‘Uniting and Strengthening American by Providing Appropriate Tools Required to Intercept and Obstruct Terorism’. According to Wikipedia, 23-year-old Congressional staffer Chris Cylke was responsible for drumming up this impressive euphemism. Big brother would be proud, Chris. After 9/11, every formerly basement dwelling American flag was dusted off and proudly flown. Going for a fall drive in 2001 was to experience a patriotic-kaleidoscopic Red White and Blue backdrop-ed against the ever-green American suburban lawn. Given the overflow of national pride at the time, there was scarcely a democratically elected lawmaker in the land with the requisite lack of regard for job security to publicly scrutinize this legislation. The USA PATRIOT Act, as it turned out, authorized unprecedented and expansive surveillance privileges to US Intelligence agencies and, despite nearly 20 years of distance between the events that spawned its creation, is still largely in place today. As a highly simplified refresher, USA PATRIOT and concurrent amendments to the Foreign Intelligence Surveillance Act (FISA) (as revealed by Snowden) resulted in two major controversial communications surveillance programs (among many other smaller programs): PRISM and XKeyscore. On one hand, the true scope of these programs is still being debated. And the accounts of their capabilities differ wildly depending on if you’re asking Snowden or the NSA. But there are enough now publicly available documents to be able to read through any face-saving cover up attempts by large, powerful intelligence agencies. PRISM officially began in 2007 after President Bush signed an amendment to FISA called the Protect America Act (the trend continues). But PRISM was effectively in place since 9/11 but was challenged as illegal in court in 2006, and so rather than pause these monitoring activities deemed unpalatable by the public courts, US intelligence opted to turn on incognito mode. PRISM meant agents for the NSA, FBI, CIA, DIA etc. could unilaterally access data and perform “ . Technically speaking, US intelligence positioned themselves behind the encryption servers of Microsoft, Google, Yahoo and many other tech companies to read data before it was ultimately encrypted and sent off to users. ” extensive, in-depth surveillance on live communications and stored information This was worse than mere surveillance. It was telling the world your data is secure via the magic of “encryption” while feeding every raw and exposed data packet right into a cavernous government database instantly and eminently searchable by thousands of agents. Such public-private covert collaboration (perhaps ‘coercement’ is a more charitable term from the perspective of tech giants) highlights the need for true peer-to-peer or end-to-end encryption instead of trusting central tech providers to manage your data and secure it. There are always conflicting demands on organizations as large as Google and Microsoft that do not always correspond to ‘Joe users’ needs and desires. Do not listen to companies who claim “end-to-end” encryption but really mean “client-to-server” encryption. For example, published a recent expose on Zoom’s data privacy practices demonstrating they actually have access to video data that sits behind the point where data gets encrypted and sent out to customers. The Verge The importance of the distinction between client-to-server and end-to-end encryption cannot be overstated. The below graphics highlight the key differences and the privacy implications that correspond to these differences. (Source: ). Wickr (Source: ). Wickr XKeyscore was an even more ambitious system that listened to and forwarded internet data at the layer of the internet’s infrastructure itself: with listening devices installed inside cross-national fiber optic cables. The specific capabilities of the system were described by Edward Snowden in an interview: ‘ .’ You could read anyone’s email in the world, anybody you’ve got an email address for. Any website: You can watch traffic to and from it. Any computer that an individual sits at: You can watch it The core capabilities of XKeyscore are analogous to the private website analytics tool ‘Hotjar’ which allows marketers to play back website sessions, seeing everything from where your mouse moved to what you clicked on. Except, instead of just operating on a single site, this capability was extended across the entire internet, and accessible live and at a moments notice by US intelligence. There are some who feel this level of surveillance is unacceptable no matter the benefits to national security. But what were the impacts of these programs that buried themselves deep inside the private lives of each and every western internet user? General Keith B. Alexander, a now retired 4-star general who served as director of the NSA cited 54 ‘ ’ as a result of information collected by surveillance programs operating under Section 215 of the Patriot Act and Section 702 of the Foreign Intelligence Surveillance Act of 1978 (FISA) (the specific programs Snowden revealed publicly in 2013). terrorist activities disrupted 54 sounds like a lot. And it is. But it’s important to consider that the tens of billions of dollars US intelligence agencies spent collecting and parsing this data was at the expense of traditional methods of anti-terrorist work. It also leads to information overload that many have pointed out makes the work of US intelligence less effective. From a 2015 with NSA Whistleblower William Binney, a former NSA official who spent more than three decades at the agency: ZDNet interview “an analyst today can run one simple query across the NSA’s various databases, only to become immediately overloaded with information…That’s why they couldn’t stop the Boston bombing, or the Paris shootings, because the data was all there,” The information overload leaves analysts looking for a needle in a haystack. The parallels between post 9/11 terrorism surveillance and present day COVID spread monitoring may seem tenous at first. But the core question of public safety vs. individual sovereignty and right to privacy remains the same. The only difference is these are public health officials instead of US intelligence officials pitching a need to access more data. Even if that data is highly personal and even sensitive. “ ” I looked at this data and thought a little bit about this being a modern day Patriot Act — a dramatic move really quick in reaction to 9/11. I think that’s kind of what you’re seeing in the public — a lot of bipartisan support for some pretty aggressive moves by the government to curb the spread of coronavirus, — John Gerzema, the chief executive of the Harris Poll. What are the programs on the table today to use mass data to increase transparency and offer insight to public health officials? In a , Raullen Chai, a Co-Founder of IoTeX called out some of the potential privacy invasions on the table: recent op-ed for the International Business Times “In Israel, the government has authorized its security service to track mobile-phone location-data of people suspected to have coronavirus using techniques originally deployed for anti-terrorism surveillance. China took advantage of facial-recognition systems to trace people’s movements in its anti-virus fight. And the United States is engaging in public-private partnerships with the likes of Palantir, a data-scraping company known for its predictive policing tools… U.S. President Donald Trump directed anxious Americans to Google’s Project Baseline, citing that it would help with coronavirus. But the Terms & Conditions state: “If you withdraw your consent, information that has already been gathered will be retained. Once you join, your membership could last indefinitely, or could be ended at any time without your permission.” Recent developments point to even more expansive surveillance steps. The $2 Trillion Coronavirus relief bill includes some $500 million for tracking and data collection, and others. The CDC must produce a report before the end of April outlining a new public health data surveillance system and modernized analytics infrastructure. as reported by Wired As quoted in , Jake Laperruque from the project on government oversight said of the initiative: “ .” Wired I could definitely see it being used to build out infrastructure for things like location tracking, cell phone tracking tools, [or] social media monitoring tools “I could definitely see it being used to build out infrastructure for things like location tracking, cell phone tracking tools, [or] social media monitoring tools.” How do Americans feel about these initiatives in light of the Snowden revelations and what many have called the increasingly privacy aware public? IoTeX ran a poll of it’s US based community, which is undoubtedly biased towards privacy conscious individuals (indicating these results may actually underrepresent the true COVID-19 related change of sentiment among Americans) and uncovered the following results. Most notably, nearly 50 % of the US based respondents said they would be comfortable letting the government track them if it meant helping reduce the spread of COVID-19. Roughly 1 / 3rd of respondents felt the same way about using a Google owned COVID-19 diagnostic test. How does this data compare with pre-COVID levels of privacy concern? Cross-referencing this data with Pew research results from June-2019, we produced the following graphic. Notably, we found an absolute increase of 13 % and a relative increase of over 63 % of the feeling that the benefits of companies collecting data on them outweighed the risks. For governments? The change was an absolute rate of 14 % or a relative change of over 41 %. Desperate times call for desperate measures. And there is no question that COVID-19 descending upon the world has brought on desperate times. The question then turns to how much of a privacy trade off is appropriate given the public health benefits? Under the PATRIOT Act, we saw how this cost-benefit analysis can get badly distorted, and in many ways the measure(s) did more harm than good even before considering the sweeping infringements on American civil liberties (namely privacy). I believe we are facing a similar cost-benefit analysis that if we are not careful, will ultimately do more harm than good. There is a false dichotomy between privacy and public health that may undermine civil liberties for years and even decades to come. Yuval Noah Harari (author of Sapiens) made the case in a in the Financial Times that: recent op-ed “ ” Asking people to choose between privacy and health is, in fact, the very root of the problem. Because this is a false choice. We can and should enjoy both privacy and health. In reality, we are confronted with scientific facts about social interactions and the requisite public health implications of these actions all the time. We are capable of making decisions as individuals without having central authorities put a gun to our heads or a tracking device around our ankles to enforce them. “Asking people to choose between privacy and health is, in fact, the very root of the problem. Because this is a false choice. We can and should enjoy both privacy and health.” — Noah Yuval Harari Ultimately, our cell phones, wearables and devices need to be end-to-end encrypted, and handed over for us to control. Government needs to trust individuals to digest scientific information and decide what makes sense for them. In other words, preserve the democratic processes that have undergirded the rise of prosperity globally over the last century. This is no time to throw away civil liberties and trust. These are the very principles that have allowed us to prosper, and we cannot abandon them when the going gets tough. WRITTEN BY Dean Patrick Previously published at https://medium.com/@deanpatrick_63570/data-privacy-in-the-time-of-coronavirus-195e0409a2c2

Privacy In The Time of Coronavirus

A Solo-Entrepreneur is a person who sets up a business venture and assumes full responsibility for the associated risk. Which traits will make one a good founder? Passion Ability to tolerate risk Leadership skills Flexibility / Adaptation Decision making Financial Management Skills Human management / relationship skills Creativity / Innovation Those traits are mostly often named as the most essential in building an enterprise from a low scale business to a multi-million-dollar company. Let’s see how 6 solo-entrepreneurs over history have changed their lives and made a positive impact on society and people around them. Andrew Carnegie and His Steel Empire Andrew Carnegie is a quintessential Gilded Age titan. He’s an industrialist and a self-made man who started from nothing and . Carnegie was born in Scotland to a working-class and skilled labor family, and his father ended up losing everything. His family moved to the United States of America penniless; thus he had no real formal education, but he spent a lot of time reading, and he is a self-taught intellect. To meet up with life’s financial challenges, Carnegie started working as a telegrapher and worked with various companies. created an empire in steel The Gilded Age was marked by an economic boom with railroads being the economic forefront, and steel is critical to the building of railroads. Carnegie recognizes this, and he saw an opportunity by the end of his career, he had a monopoly on the production of steel in the United States. So, by 1901, Andrew Carnegie is one of the richest men in the world and currently has a net worth of $310 billion. Aside being the sole-proprietor of Carnegie Steel Company, Carnegie went on to set up several enterprises like: Carnegie institution of science Carnegie library Carnegie Corporation of New York Carnegie Endowment for international peace Carnegie Mellon university Carnegie hero fund Markus Frind and His Dating App Markus Frind is an internet entrepreneur based in Germany, primarily known as the creator of . Plenty of Fish as it was initially known was established in 2003, and since its creation it has risen to become one of the most in-demand dating sites with active users in Canada, Brazil, USA, Australia, Ireland, and the United Kingdom. popular dating social network POF A graduate in computer systems technology from the prestigious British Columbia Institute of Technology (1999), he worked with several online companies and held several positions like system analyst, cybersecurity and web developer. After several unsuccessful jobs Frind followed his long ambition of being self-employed. He created POF in 2003 and by July 2013 made a profit of $1100. Now, POF serves a minimum of 1.5 billion webpages monthly and generates $100 million a year. Vishen Lakhiani and Education Technology Born 14th January 1976 in Kuala Lumpur, Malaysia with lots of complications ranging from personal (poor eyesight, Asperger syndrome, skin complications) to amenities (complicated schooling system). Vishen fought through all these barriers and is currently a , an author of many books with his New York bestseller “The Code of the Extraordinary Mind” selling Hundred thousand copies worldwide, most importantly he is an entrepreneur and founder of 3 companies (Mindvalley, Dealmates, and Blinklist). speaker with millions of active subscribers His primary company, Mindvalley, deals with education technology, spiritual development, and publishing of daily self-care software such as Omvana. Mindvalley was built with $700 and is now worth over $120 million. In 2015, the 43-year-old Malaysian won the award for “Best New Idea” at the XPRIZE’s Visioneering event held in Culver City, California. Ingvar Kamprad and Furniture Paradise Ingvar Feodor Kamprad, a Swedish business tycoon, popularly known for founding IKEA, an intercontinental furniture trade company. Born March 30th 1926 in Sweden, Kamprad displayed his entrepreneurship traits from an early age when he began selling Matchbox in his neighborhood, then upgraded to selling nylons, picture frames, stockings, and pieces of jewelry. In 1943, at just 17 years, he (a combination of his initials IK, the farm he grew up in ELMTARYD, and his village AGUNNARYD) which turned out to be the world’s biggest retailer of furniture. founded IKEA In 1948 Kmaprad stopped other retail and focused chiefly on inexpensive furniture and offer home furnishing services, he expanded to renting a showroom in Sweden. Kamprad sold quality furniture at an amazingly low price which led to fast-rising popularity amongst the Swedish market. IKEA introduced flat boxed furniture in 1956 which led to a significant breakthrough and attracted interested buyers all over Europe and some parts of the world. Kamprad went from being a childhood retailer to a global business Moghul worth over $58.7 billion. Jaseel SK and Cheap Flights Affiliate marketing still shines on the internet, even if it doesn’t have quite the buzz it used to. The key is to remain persistent. Many people try their hand at affiliate marketing but leave after a short amount of time if they aren’t making money. Every day, Jaseel started earning a bit more through the Travelpayouts program. He didn’t know anything about the online market, and it took me months and long hours to get to a place where he was earning a profit. Later, he found out about Travelpayouts’ mobile SDK for both Android and iOS and he used that to develop the first version of an app for both platforms. Shortly after, he . had the app published on both platforms Until 2018, Jaseel was slightly over $200K USD per year from Travelpayouts, and most of the income came from mobile apps. He finished college in 2018 and fully focused on online marketing opportunities. Now, he still lives in India, so it’s a pretty high income. He said he could only imagine such income from his own venture and no corporate office job. Daymond John and Hip Hop Clothing Fashion designer, Daymond John, founder of famous was born in New York. His entrepreneurial passion started at a young age as he sold almost everything he could spare from Pencils to reconditioned Toy cars. John worked as a waiter at Red Lobsters long before starting his clothing brand. Hip Hop clothing brand FUBU In 1992, John kicked off business by making hats in his mother’s basement and selling them off at concerts and festivals around his neighborhood, in a bid to boost his business John mortgage his house and invested all the money into his business. FUBU an acronym meaning For Us By Us expanded their collection from just hats to the inclusion of-shirts, Hockey Jerseys, and Baseball caps which all had FUBU label embroidered in them. John’s brand skyrocketed when entertainer LL Cool J supported him by creating awareness about the brand which quickly rose to become the trending urban clothing, setting fashion trends for African American youths and most lovers of Hip Hop. FUBU foundation was created in 1997 by John, and it raised $1 million yearly which was donated to charity. Proud of his achievements, John inflated his business realm with the launching of FUBU Entertainment and FUBU Records which released a 19-track musical album which helped increased popularity of the FUBU brand and increased the sale to over $100 million during its launch. Daymond John is estimated to worth around $400 million.

Stories, News & Mentions about DEALPATH

Dealpath’s Stories on HackerNoon