Table of Contents Introduction Key Takeaway Facts about OmegaPro Ltd Services Offered by OmegaPro Ltd Benefits of Choosing OmegaPro Ltd Frequently Asked Questions (FAQs) Introduction OmegaPro Ltd is a leading name in the world of forex trading and investment. With its cutting-edge technology and experienced team, the company has gained a reputation for providing innovative […]

OmegaPro Ltd: A Revolutionary Forex Company

OmegaPro World is a renowned online trading platform that specializes in forex trading, offering individuals the opportunity to profit from global currency exchange markets. With a secure and user-friendly platform, OmegaPro World provides users with ...

OmegaPro World Forex Trading: A Comprehensive Guide

Forex Trading Overview Key Features of OmegaPro Website Different Investment Plans Effective Risk Management Strategies High-Level Security Measures Frequently Asked Questions ## Introduction {#intro} Forex trading has evolved into one of the most lucrative investment opportunities, attracting both professional investors and individuals seeking financial freedom.

OmegaPro Website: An In-Depth Review of Forex Trading Opportunities

However, navigating the complexities of the Forex market requires expertise and the right tools. OmegaPro Forex is a revolutionary platform that aims to empower traders by providing them with a comprehensive set of trading tools, education, and support.

OmegaPro Forex: Unlocking Profits in the Forex Market

There are also some people who will also charge a fee to act as an agent, asking you to pay to obtain the same website, and then promote and charge other people's membership fees. The routine has ...

Omegapro Investment ️️ Discovery Invest

Atoms: Pull in the best mechanisms to solve challenging problems in the physical world. Those friends who surround you all day and make you a little happy are not necessarily true friends. In addition, data governance will receive more and more attention ...

Governor Bill Lee Website Hacked - Bitcoin Vpn check wallet bitcoin

How To Do Compounding Investment - Omegapro Investment

Jordan Belfort, the "Wolf of Wall Street," rose to infamy due to his participation in securities fraud and financial crimes during the 1990s.

‘Wolf of Wall Street’ J. Belfort responds to Ponzi scheme allegations

“Final Fantasy 16 is absolutely stunning,” OmegaPro wrote on Twitter. Kurrie said: “It has been a pleasure knowing everyone, but the moment I get home, Final Fantasy 16 will take me.”

‘Absolutely stunning’: Final Fantasy 16 drops and fans are loving it

OmegaPro steers the world towards a better future with the help of world-renowned motivational speakers Les Brown, John C. Maxwell, Eric Thomas and Eric Worre. “Knowledge is power. Information ...

OmegaPro Lays Down The Success Pathway Through World-Class Motivational Speakers

OmegaPro

omegapro

RESTful APIs are everywhere, powering more of the modern world than any other API architecture. REST accounts for up to 80% of APIs . How those APIs are built and structured can make or break a company in today’s ultra-competitive world. Poorly-designed APIs can be difficult to use, can fail when they are needed most, and are valuable targets for hackers looking for sensitive data. On the other hand, a well-designed API that utilizes best practices makes development a breeze, attracts new customers, and creates confidence among users that can boost retention rates. according to this study by ProgrammableWeb What is a RESTful API? REST is an acronym for presentational tate ransfer, and describes an architectural style for creating distributed web services. REST allows users to use standard HTTP requests to remotely call code and receive responses. REST has several advantages over other protocols: RE S T It separates data storage concerns from the user interface, meaning a back-end server running an API can handle all of the logic to access databases and manipulate data before returning it to a user in a uniform, structured way. This consistent access and structure of data allows developers to build front-end applications easily, which makes porting the front end of your app to other platforms a breeze.REST APIs support caching of commonly requested static resources, lending to better performance.REST is stateless, so all information to complete the request is included in the request. This simplifies the API by removing the need for server-side state synchronization logic. This also makes scaling easier, as any server can handle any request without tracking sessions. (If building RESTful APIs is new to you, or you want a sandbox to try out these best practices, try out this from Heroku that is quick and easy to deploy.) reference architecture So now, here are the top five best practices for building your RESTful APIs. 1. Use Error Status Codes HTTP has already built in. Using status codes in your RESTful API to communicate the general error is a godsend for developers. Developers will be able to immediately identify the issue, which means spending less time writing parsers to handle all the different types of errors. Is the request being denied because the session is not logged in? There’s a status code for that. over 100 status codes Is there a missing resource? Did the CRUD service receive the request but fail to connect to the database? There are status codes for those, and nearly every other common behavior. Status codes can also be combined with specific error messages to provide detailed information about unsuccessful requests. Here’s a snippet of code from a Node.js API to illustrate: This endpoint returns a user profile by the passed-in user ID. The status codes sent in response to the requests tell developers exactly what error happened, making the response easier to handle and saving them significant time and trouble. Developers can implement routines for handling different errors based on status codes, while the API provides detailed error information. In this case, the 404 error tells the caller that something can’t be found. The JSON in the response tells the caller specifically that it’s the user ID that can’t be found, rather then being ambiguous about whether the error refers to the endpoint or requested resource. 2. Good Documentation Documentation is one of the most important — and most overlooked — aspects of an API. Official documentation can be a customer’s first point of contact with a product, and a key factor in whether or not a development team adopts it. Good documentation looks clean and consistent, and adequately prepares a developer to use your API quickly. The faster someone can learn your API, the faster they can start producing with it. Documentation should have a uniform look, and include all of the relevant information: the endpoint, compatible methods (GET, POST, PUT, etc.), what parameters are optional and required, and the data type expected. This screenshot from illustrates what complete documentation for developers looks like. It shows the action taken, the endpoint accessed, and the HTTP method used. It also supplies detailed information about the optional parameters and shows the user a working example of everything implemented correctly. The sample responses also illustrate how the returned data will be structured. Heroku’s platform API documentation 3. Rate Limiting and Throttling API requests can be resource intensive, requiring serious computing power and storage. If you aren’t careful, a large number of successive, concurrent requests can slow down or even DOS your server.An easy way to do this is to use one of the many available tools like , an Express middleware designed specifically to handle Rate Limiting in an easy, intuitive way. express-rate-limit You can also implement rate limiting logic tied to authentication, allowing greater flexibility in controlling permissions granted to each user. By requiring users to authenticate, it’s possible to track the number of requests each users sends, which also allows you to limit or stop those requests. Different users can also be granted access to different API endpoints. For example, a user who is an Administrator could access more information, or more requests, from an API than a regular, unprivileged user. Another benefit of using authentication is the security it provides, bringing us to our next best practice. 4. Secure The API APIs need to be secure! Hackers use automated scripts to attack services indiscriminately, so an API to have to keep operations running smoothly and to protect sensitive data. First and foremost, every web application should have a TTP trict ransport ecurity (HSTS) policy to ensure all connections are encrypted. Securing the connection prevents network sniffing, man-in-the-middle attacks, protocol downgrade attacks, and session hijacking via cookie theft. needs proactive security measures H S T S You may also want to set and hide certain headers that can be exploited, such as those that reveal information about your API infrastructure that may be useful to attackers. There are many tools out there that can handle this. For instance, if you’re running an API with Node.js, you can use something like . Implementing this middleware is easy: Helmet.js To prevent an API from leaking sensitive customer data, such as passwords, for security testing. Last but not least, you should require an authentication token to access your API. This allows developers to control who has access to what information. It can also make it easier to stop attacks on the API server by denying offending users. write unit tests 5. Use JSON The purpose of an API is to serve data from your company’s resources. There are three formats that are commonly used to return the data : XML, YAML, and JSON. according to Nordic APIs XML is easily human-readable, but the data is contained within a set of markup tags, which quickly grows in size and requires extra bandwidth. Developers also have to parse the contents of the tags to access the data.YAML, by contrast, uses very little bandwidth, but requires either an external library or a custom parser and encoder to work with the data.JSON beautifully marries the worlds of XML and YAML together: it’s human-readable without needing high bandwidth or custom parsing to move the data into a JavaScript-compatible structure. Conclusion Implementing these five practices with your RESTful APIs will make your API easier and safer to use. A safe, secure API with good documentation can deliver a great developer experience, and the added ease of use will help your adoption rates. Furthermore, these best practices will keep your code clean, and your operations running smoothly, and your customers happy. Previously published at https://dev.to/omegapointzero/5-best-practices-for-designing-restful-apis-331k

5 Best Practices for Designing RESTful APIs

Machine learning experiments require extensive parametrization, including optimizer parameters, network architecture, and data augmentation. However, we strive for concise, readable code instead of a cumbersome 200 lines dedicated to argparse. Our goal is to focus on programming logic rather than threading new parameters through function signatures. Additionally, we seek a structure that is easily expandable without burdening the project, ensuring the reproducibility of experiments. Hydra offers a solution to these challenges. Below, you will find a basic guide on how to use it. What is Hydra? Hydra is a library with rich capabilities for managing configurations. The main site describes the name like this: “The key feature is the ability to dynamically create a hierarchical configuration by composition and override it through config files and the command line.The name Hydra comes from its ability to run multiple similar jobs - much like a Hydra with multiple heads.” But I have my own interpretation of its name. It is just a combination of some tools that work together impressively well! Hydra offers a seamless solution to the common headaches faced by ML engineers when attempting to replicate experiments. It elegantly replaces the need for argparse or YAML configurations, allowing access to parameters both from the command line and YAML files. Consider the pain points: Replicating experiments using argparse forces reliance on string inputs and complicates YAML config launches. Relying solely on YAML files for configuration leads to duplication and potential errors when only a single parameter needs alteration. Hydra addresses these issues by enabling dynamic configuration adjustments without the need for multiple bulky files or rigid command-line arguments. Furthermore, it simplifies the process of passing complex configurations, such as model architectures or functions, directly from the config file to the model. This capability eliminates the tedious task of manually feeding parameters into the model, streamlining the workflow and reducing the margin for error. Basic Setup Let's imagine the simplest setup: multiclass classification on MNIST using an MLP. We have a configuration and a training script. .
├── configs
│   └── config.yaml
└── main.py The main script can look like this. import torch
import torch.nn as nn
import torch.optim as optim
from torchvision import datasets, transforms
from torch.utils.data import DataLoader

import hydra
from omegaconf import DictConfig

class Net(nn.Module):
    def __init__(self):
        super(Net, self).__init__()
        self.model = nn.Sequential(
            nn.Linear(28*28, 128),
            nn.ReLU(),
            nn.Linear(128, 64),
            nn.ReLU(),
            nn.Linear(64, 10)
        )
    
    def forward(self, x):
        return self.model(x.view(-1, 28*28))

@hydra.main(version_base=None, config_path=None, config_name="config")
def main(cfg: DictConfig):
    # Load MNIST dataset
    transform = transforms.Compose([
        transforms.ToTensor(), 
        transforms.Normalize((0.5,), (0.5,))
    ])
    train_dataset = datasets.MNIST(root='./data', train=True, download=True, transform=transform)
    train_loader = DataLoader(train_dataset, batch_size=cfg.batch_size, shuffle=True)

    # Initialize the network, loss function, and optimizer
    model = Net()
    criterion = nn.CrossEntropyLoss()
    optimizer = optim.SGD(model.parameters(), lr=cfg.lr, momentum=cfg.momentum)

    # Train the network
    for epoch in range(cfg.epochs):  # loop over the dataset multiple times
        for i, (inputs, labels) in enumerate(train_loader, 0):
            optimizer.zero_grad()  # zero the parameter gradients
            outputs = model(inputs)  # forward pass
            loss = criterion(outputs, labels)  # calculate loss
            loss.backward()  # backward pass
            optimizer.step()  # optimize

    print('Finished Training')

if __name__ == "__main__":
    main() The configuration has the following structure. It is the default, and parameters from it will be used by default if not specified otherwise. # conf/config.yaml
batch_size: 64
lr: 0.01
momentum: 0.9
epochs: 1 To start training you can use: python main.py And for changing parameters, you can do it not only from YAML but also from cli: python main.py lr=0.03 If you use it, parameters from cli override the first parameters. As you can see, there is no argparse or additional middleware. It's easy to change parameters from the command line. Pass Class Object With Hydra We see that in class Net, the architecture can be highly customizable. Even more, we can use also another net for this, e.g. CNN. But we do not want to change our pipeline. Hydra can construct almost any Python object with specified parameters. Let’s describe our net in another way, using YAML. model:
  _target_: torch.nn.Sequential
  layers:
    - _target_: torch.nn.Flatten
    - _target_: torch.nn.Linear
      in_features: 784  # 28x28 images are flattened into 784
      out_features: 128
    - _target_: torch.nn.ReLU
    - _target_: torch.nn.Linear
      in_features: 128
      out_features: 64
    - _target_: torch.nn.ReLU
    - _target_: torch.nn.Linear
      in_features: 64
      out_features: 10 and for CNN _target_: torch.nn.Sequential
  layers:
    - _target_: torch.nn.Conv2d
      in_channels: 1  # MNIST images are grayscale, so 1 input channel
      out_channels: 32  # Number of output channels
      kernel_size: 3  # Size of the convolutional kernel
      stride: 1
      padding: 1
    - _target_: torch.nn.ReLU
    - _target_: torch.nn.MaxPool2d
      kernel_size: 2  # Pooling window size
      stride: 2
    - _target_: torch.nn.Conv2d
      in_channels: 32
      out_channels: 64
      kernel_size: 3
      stride: 1
      padding: 1
    - _target_: torch.nn.ReLU
    - _target_: torch.nn.MaxPool2d
      kernel_size: 2
      stride: 2
    - _target_: torch.nn.Flatten  # Flatten the output for the fully connected layer
    - _target_: torch.nn.Linear
      in_features: 7*7*64  # Size after convolutions and pooling
      out_features: 128
    - _target_: torch.nn.ReLU
    - _target_: torch.nn.Dropout
      p: 0.5  # Dropout rate
    - _target_: torch.nn.Linear
      in_features: 128
      out_features: 10  # Number of classes in MNIST Pass Function Through Config Sometimes, our parameters can also function. For example, How to Access Config Without Decorating Main - e.g., in ipython Notebook. So, you can use Compose API. from hydra import compose, initialize from omegaconf import OmegaConf

with initialize(version_base=None, config_path="conf", job_name="run_0001"):
  cfg = compose(config_name="config", overrides= ["parameter=value"])

print (OmegaConf.to_yaml(cfg)) Multi Runs For example, you want to find some good hyperparam and Hydra allows a lot for that. Basically, you can launch several experiments sequentially like. python train_model.py -m "batch_size=16,32,64" It launches sequentially, but it is easy to extend to parallel if turned on some launcher like joblib. python train_model.py -m "batch_size=16,32,64 hydra/launcher=joblib" Run All Experiments From the Folder for config_file in configs/*.yaml; do python main.py --config-name="${config_file}"; done Resources hydra - https://hydra.cc/
good template for DL - https://github.com/ashleve/lightning-hydra-template/tree/main Machine learning experiments require extensive parametrization, including optimizer parameters, network architecture, and data augmentation. However, we strive for concise, readable code instead of a cumbersome 200 lines dedicated to argparse. Our goal is to focus on programming logic rather than threading new parameters through function signatures. Additionally, we seek a structure that is easily expandable without burdening the project, ensuring the reproducibility of experiments. Hydra offers a solution to these challenges. Below, you will find a basic guide on how to use it. What is Hydra? Hydra is a library with rich capabilities for managing configurations. The main site describes the name like this: “The key feature is the ability to dynamically create a hierarchical configuration by composition and override it through config files and the command line.The name Hydra comes from its ability to run multiple similar jobs - much like a Hydra with multiple heads.” “The key feature is the ability to dynamically create a hierarchical configuration by composition and override it through config files and the command line. The name Hydra comes from its ability to run multiple similar jobs - much like a Hydra with multiple heads.” But I have my own interpretation of its name. It is just a combination of some tools that work together impressively well! Hydra offers a seamless solution to the common headaches faced by ML engineers when attempting to replicate experiments. It elegantly replaces the need for argparse or YAML configurations, allowing access to parameters both from the command line and YAML files. Consider the pain points: Replicating experiments using argparse forces reliance on string inputs and complicates YAML config launches. Replicating experiments using argparse forces reliance on string inputs and complicates YAML config launches. Relying solely on YAML files for configuration leads to duplication and potential errors when only a single parameter needs alteration. Relying solely on YAML files for configuration leads to duplication and potential errors when only a single parameter needs alteration. Hydra addresses these issues by enabling dynamic configuration adjustments without the need for multiple bulky files or rigid command-line arguments. Furthermore, it simplifies the process of passing complex configurations, such as model architectures or functions, directly from the config file to the model. This capability eliminates the tedious task of manually feeding parameters into the model, streamlining the workflow and reducing the margin for error. Basic Setup Let's imagine the simplest setup: multiclass classification on MNIST using an MLP. We have a configuration and a training script. .
├── configs
│   └── config.yaml
└── main.py .
├── configs
│   └── config.yaml
└── main.py The main script can look like this. import torch
import torch.nn as nn
import torch.optim as optim
from torchvision import datasets, transforms
from torch.utils.data import DataLoader

import hydra
from omegaconf import DictConfig

class Net(nn.Module):
    def __init__(self):
        super(Net, self).__init__()
        self.model = nn.Sequential(
            nn.Linear(28*28, 128),
            nn.ReLU(),
            nn.Linear(128, 64),
            nn.ReLU(),
            nn.Linear(64, 10)
        )
    
    def forward(self, x):
        return self.model(x.view(-1, 28*28))

@hydra.main(version_base=None, config_path=None, config_name="config")
def main(cfg: DictConfig):
    # Load MNIST dataset
    transform = transforms.Compose([
        transforms.ToTensor(), 
        transforms.Normalize((0.5,), (0.5,))
    ])
    train_dataset = datasets.MNIST(root='./data', train=True, download=True, transform=transform)
    train_loader = DataLoader(train_dataset, batch_size=cfg.batch_size, shuffle=True)

    # Initialize the network, loss function, and optimizer
    model = Net()
    criterion = nn.CrossEntropyLoss()
    optimizer = optim.SGD(model.parameters(), lr=cfg.lr, momentum=cfg.momentum)

    # Train the network
    for epoch in range(cfg.epochs):  # loop over the dataset multiple times
        for i, (inputs, labels) in enumerate(train_loader, 0):
            optimizer.zero_grad()  # zero the parameter gradients
            outputs = model(inputs)  # forward pass
            loss = criterion(outputs, labels)  # calculate loss
            loss.backward()  # backward pass
            optimizer.step()  # optimize

    print('Finished Training')

if __name__ == "__main__":
    main() import torch
import torch.nn as nn
import torch.optim as optim
from torchvision import datasets, transforms
from torch.utils.data import DataLoader

import hydra
from omegaconf import DictConfig

class Net(nn.Module):
    def __init__(self):
        super(Net, self).__init__()
        self.model = nn.Sequential(
            nn.Linear(28*28, 128),
            nn.ReLU(),
            nn.Linear(128, 64),
            nn.ReLU(),
            nn.Linear(64, 10)
        )
    
    def forward(self, x):
        return self.model(x.view(-1, 28*28))

@hydra.main(version_base=None, config_path=None, config_name="config")
def main(cfg: DictConfig):
    # Load MNIST dataset
    transform = transforms.Compose([
        transforms.ToTensor(), 
        transforms.Normalize((0.5,), (0.5,))
    ])
    train_dataset = datasets.MNIST(root='./data', train=True, download=True, transform=transform)
    train_loader = DataLoader(train_dataset, batch_size=cfg.batch_size, shuffle=True)

    # Initialize the network, loss function, and optimizer
    model = Net()
    criterion = nn.CrossEntropyLoss()
    optimizer = optim.SGD(model.parameters(), lr=cfg.lr, momentum=cfg.momentum)

    # Train the network
    for epoch in range(cfg.epochs):  # loop over the dataset multiple times
        for i, (inputs, labels) in enumerate(train_loader, 0):
            optimizer.zero_grad()  # zero the parameter gradients
            outputs = model(inputs)  # forward pass
            loss = criterion(outputs, labels)  # calculate loss
            loss.backward()  # backward pass
            optimizer.step()  # optimize

    print('Finished Training')

if __name__ == "__main__":
    main() The configuration has the following structure. It is the default, and parameters from it will be used by default if not specified otherwise. # conf/config.yaml
batch_size: 64
lr: 0.01
momentum: 0.9
epochs: 1 # conf/config.yaml
batch_size: 64
lr: 0.01
momentum: 0.9
epochs: 1 To start training you can use: python main.py python main.py And for changing parameters, you can do it not only from YAML but also from cli: python main.py lr=0.03 python main.py lr=0.03 If you use it, parameters from cli override the first parameters. As you can see, there is no argparse or additional middleware. It's easy to change parameters from the command line. Pass Class Object With Hydra We see that in class Net, the architecture can be highly customizable. Even more, we can use also another net for this, e.g. CNN. But we do not want to change our pipeline. Hydra can construct almost any Python object with specified parameters. Let’s describe our net in another way, using YAML. model:
  _target_: torch.nn.Sequential
  layers:
    - _target_: torch.nn.Flatten
    - _target_: torch.nn.Linear
      in_features: 784  # 28x28 images are flattened into 784
      out_features: 128
    - _target_: torch.nn.ReLU
    - _target_: torch.nn.Linear
      in_features: 128
      out_features: 64
    - _target_: torch.nn.ReLU
    - _target_: torch.nn.Linear
      in_features: 64
      out_features: 10 model:
  _target_: torch.nn.Sequential
  layers:
    - _target_: torch.nn.Flatten
    - _target_: torch.nn.Linear
      in_features: 784  # 28x28 images are flattened into 784
      out_features: 128
    - _target_: torch.nn.ReLU
    - _target_: torch.nn.Linear
      in_features: 128
      out_features: 64
    - _target_: torch.nn.ReLU
    - _target_: torch.nn.Linear
      in_features: 64
      out_features: 10 and for CNN _target_: torch.nn.Sequential
  layers:
    - _target_: torch.nn.Conv2d
      in_channels: 1  # MNIST images are grayscale, so 1 input channel
      out_channels: 32  # Number of output channels
      kernel_size: 3  # Size of the convolutional kernel
      stride: 1
      padding: 1
    - _target_: torch.nn.ReLU
    - _target_: torch.nn.MaxPool2d
      kernel_size: 2  # Pooling window size
      stride: 2
    - _target_: torch.nn.Conv2d
      in_channels: 32
      out_channels: 64
      kernel_size: 3
      stride: 1
      padding: 1
    - _target_: torch.nn.ReLU
    - _target_: torch.nn.MaxPool2d
      kernel_size: 2
      stride: 2
    - _target_: torch.nn.Flatten  # Flatten the output for the fully connected layer
    - _target_: torch.nn.Linear
      in_features: 7*7*64  # Size after convolutions and pooling
      out_features: 128
    - _target_: torch.nn.ReLU
    - _target_: torch.nn.Dropout
      p: 0.5  # Dropout rate
    - _target_: torch.nn.Linear
      in_features: 128
      out_features: 10  # Number of classes in MNIST _target_: torch.nn.Sequential
  layers:
    - _target_: torch.nn.Conv2d
      in_channels: 1  # MNIST images are grayscale, so 1 input channel
      out_channels: 32  # Number of output channels
      kernel_size: 3  # Size of the convolutional kernel
      stride: 1
      padding: 1
    - _target_: torch.nn.ReLU
    - _target_: torch.nn.MaxPool2d
      kernel_size: 2  # Pooling window size
      stride: 2
    - _target_: torch.nn.Conv2d
      in_channels: 32
      out_channels: 64
      kernel_size: 3
      stride: 1
      padding: 1
    - _target_: torch.nn.ReLU
    - _target_: torch.nn.MaxPool2d
      kernel_size: 2
      stride: 2
    - _target_: torch.nn.Flatten  # Flatten the output for the fully connected layer
    - _target_: torch.nn.Linear
      in_features: 7*7*64  # Size after convolutions and pooling
      out_features: 128
    - _target_: torch.nn.ReLU
    - _target_: torch.nn.Dropout
      p: 0.5  # Dropout rate
    - _target_: torch.nn.Linear
      in_features: 128
      out_features: 10  # Number of classes in MNIST Pass Function Through Config Sometimes, our parameters can also function. For example, How to Access Config Without Decorating Main - e.g., in ipython Notebook. So, you can use Compose API . Compose API from hydra import compose, initialize from omegaconf import OmegaConf

with initialize(version_base=None, config_path="conf", job_name="run_0001"):
  cfg = compose(config_name="config", overrides= ["parameter=value"])

print (OmegaConf.to_yaml(cfg)) from hydra import compose, initialize from omegaconf import OmegaConf

with initialize(version_base=None, config_path="conf", job_name="run_0001"):
  cfg = compose(config_name="config", overrides= ["parameter=value"])

print (OmegaConf.to_yaml(cfg)) Multi Runs For example, you want to find some good hyperparam and Hydra allows a lot for that. Basically, you can launch several experiments sequentially like. python train_model.py -m "batch_size=16,32,64" python train_model.py -m "batch_size=16,32,64" It launches sequentially, but it is easy to extend to parallel if turned on some launcher like joblib. python train_model.py -m "batch_size=16,32,64 hydra/launcher=joblib" python train_model.py -m "batch_size=16,32,64 hydra/launcher=joblib" Run All Experiments From the Folder for config_file in configs/*.yaml; do python main.py --config-name="${config_file}"; done for config_file in configs/*.yaml; do python main.py --config-name="${config_file}"; done Resources hydra - https://hydra.cc/ good template for DL - https://github.com/ashleve/lightning-hydra-template/tree/main hydra - https://hydra.cc/ https://hydra.cc/ good template for DL - https://github.com/ashleve/lightning-hydra-template/tree/main https://github.com/ashleve/lightning-hydra-template/tree/main

How to Configure Experiments With Hydra - From an ML Enginner Perspective

Last year, the Uber Engineering team on their new load-shedding mechanism designed for their published an article microservice architecture. This article is very interesting from various perspectives. So, I took some notes as I was reading it to capture my understanding and write down things that I would like to delve deeper into later if I don’t get the answers by the end. I’ve found multiple times that this is the best way for me to learn new things. What got me from the start was the reference to century-old principles used to build this solution. That’s something I love — borrowing concepts/ideas from different fields and adapting them to solve a problem in a different domain. If system resiliency and stability are of interest to you, I recommend also reading the excellent book by Michael T. Nygard. ‘Release It!’ It’s an oldie but a goodie — a book that delves into strategies, patterns, and practical guidance for building resilient and stable software systems, emphasizing how to handle failures effectively. Uber has implemented a new load shedding solution called Cinnamon that takes advantage of a PID controller (the centuries-old mechanism) to decide which requests should be processed or discarded by a service based on the service’s current load and the request priority. It does not involve any tuning at the service level (although I had a question about that), is automatically adaptable, and much more efficient than their previous solution QALM. Remember also that Uber’s microservices architecture is not for the faint of heart… What is this famous PID controller and how is it used? A PID controller is an instrument used in industrial control applications to regulate temperature, flow, pressure, speed, and other process variables. PID (proportional integral derivative) controllers use a control loop feedback mechanism to control process variables and are the most accurate and stable controllers. — https://www.omega.co.uk/prodinfo/pid-controllers.html If you want more information about this centuries-old concept, head to Wikipedia. Now, back to the article. PID stands for Proportional, Integral, and Derivative. In their case, they use a component known as a PID controller to monitor the health of a service (input requests) based on three components (or measures). Proportional The term “proportional” indicates that the action taken is proportional to the current error. In simple terms, this means that the correction applied is directly proportional to the difference between the desired state and the actual state. If the error is large, the corrective action is proportionally large. When an endpoint is overloaded, the background goroutine starts to monitor the inflow and outflow of requests into the priority queue. So, the Proportional (P) component in the load shedder adjusts the shedding rate based on how far the current queue size is from the target or desired queue size. If the queue is larger than desired, more shedding occurs; if it’s smaller, shedding is reduced. That’s my understanding of it. Integral The PID controller’s job is to minimize the number of queued requests, whereas the auto-tuner’s job is to maximize the throughput of the service, without sacrificing the response latencies (too much). While the text doesn’t explicitly mention “Integral (I)” in the context of the queue size, it indicates that the PID controller’s role is to minimize the number of queued requests. The minimization of queued requests aligns with the Integral component’s goal of addressing accumulated errors over time. To determine whether an endpoint is overloaded we keep track of the last time the request queue was empty, and if it has not been emptied in the last say 10 seconds we deem the endpoint to be overloaded (inspired by Facebook). In the load shedder, it may be associated with decisions related to the historical behavior of the request queue, such as the time since it was last empty. Honestly, that’s not entirely clear to me. It’s a bit frustrating, I must say. While they mention leveraging a centuries-old mechanism, it would have been helpful if they explicitly stated which part corresponds to what or how it operates. I do not want to diminish the value of their amazing article. That’s just my rant here… After all, I’m French… ;) Derivative I think this one is easier to identify. In a classical PID (Proportional-Integral-Derivative) controller, the “Derivative (D)” action is particularly useful when you want the controller to anticipate the future behavior of the system based on the current rate of change of the error. It helps dampen oscillations and improve the system’s stability. In the context of the load shedder and the PID controller mentioned in the article, the Derivative component is likely employed to assess how fast the request queue is filling up. By doing so, it aids in making decisions that aim to maintain a stable system and prevent sudden or unpredictable changes. The rejector component has two responsibilities: a) figure out whether an endpoint is overloaded and b), if an endpoint is overloaded, shed a percentage of the requests to make sure the request queue is as small as possible. When an endpoint is overloaded, the background goroutine starts to monitor the inflow and outflow of requests into the priority queue. Based on these numbers, it uses a PID controller to determine a ratio of requests to shed. The PID controller is very fast (as in very few iterations are needed) at finding the correct level and once the request queue has been drained, the PID ensures that we only slowly reduce the ratio. In the mentioned context, the PID controller is used to determine the ratio of requests to shed when an endpoint is overloaded, and it monitors the inflow and outflow of requests. The derivative component of the PID controller, which responds to the rate of change, is implicitly involved in assessing how fast the request queue is being filled or drained. This helps in making dynamic decisions to maintain system stability. Both the Integral and Derivative components are involved in monitoring the behavior of the request queue over time Integral Component In the context of determining overload, the integral component might be associated with tracking how long the request queue has been in a non-empty state. This aligns with the idea of accumulating the integral of the error signal over time. “Integral — based on how long the request has been in the queue…” Derivative Component The derivative component, on the other hand, is related to the rate of change. It responds to how quickly the state of the request queue is changing. “Derivative — rejection based on how fast the queue is filling up…” While both Integral and Derivative aspects involve observing the request queue, they focus on different aspects of its behavior The Integral component emphasizes the duration of the non-empty state, while the Derivative component considers the rate at which the queue is changing. At the end of the game, they utilize these three measures to determine the course of action for a request. The question I have is how they combine these three components, if at all. I’m curious to understand how they monitor them as well. Nevertheless, I think I’ve got the idea… Which request do they decide to shelve depending on these 3 components, and how? The endpoint in the edge is annotated with the priority of the request and this is propagated from the edge to all downstream dependencies via . By propagating this information, all services in the request chain will know the importance of the request and how critical it is for our users. Jaeger The first thought that comes to mind is that it would seamlessly integrate into a service mesh architecture. I appreciate the concept of employing distributed service tracing and headers to propagate request priority. Along these lines, why opt for a shared library with this dependency added to each microservice, instead of placing it outside the service, perhaps as an Istio plugin? Considering the benefits it offers: independent release/deployment cycles, polyglot support, etc. Here are some additional thoughts: Is there a possibility that, over time, the request priority may degrade or change based on contextual shifts, such as the initiation of an unplanned high-priority batch process or task, or alterations at the timezone/region level? How does this approach play out with other ‘standard’ resiliency management approaches/patterns, such as circuit breakers, timeouts, leases, bulkheads, etc.? I’m interested in understanding to what extent a service decides (owns the decision) to shed requests based on the input request priority. They mention a lack of configuration (an aspect I would like to explore further), so it’s not specific to the service and entirely independent of the service implementation, kind of transparent. This brings back the point of service mesh/independent release cycles versus dependent library. Well, I’m biased, as I’m not a big fan of shared libraries, if only because I think they complicate the release/deployment process. However, I’m not sure if there is a service-specific configuration aspect to consider. Perhaps they configure how long the service should wait to start processing a query and complete it? Perhaps one aspect worth testing is the decision-making process of the ejector. From what I understand, it determines whether to reject a request based on the PID controller, which is localized to the service. Is there an option for a more global approach? For instance, if it’s known that one of the downstream services in the pipeline is overloaded (due to its own PID controller), could any upstream service decide to reject the request before it reaches this overloaded service (which could be n steps further down the path)? This decision could be based on the value returned by the PID controller or the auto-tuner of the downstream service. Now, I’m musing on various aspects mentioned as they wrap up the article and provide some numbers to showcase the efficiency of their system, which is quite impressive They mention at some point that ‘Each request has a timeout of 1 second.’ We run tests of 5 minutes, where we send a fixed amount of RPS (e.g., 1,000), where 50% of the traffic is tier 1 and 50% is tier 5. Each request has a timeout of 1 second. It is common in distributed systems to associate a request with a specific expiration time or deadline, with each service along the processing path responsible for enforcing this time limit. If the expiration time is reached before the request is completed, any service in the chain has the option to abort or reject the request. I assume this 1-second timeout is attached to the request, and each service, depending on where we are in this deadline, can decide to abort the request. This is a measure that is global because it is aggregated through the services. I think it resonates with the point I was making earlier about having a global view of the full system health or dependencies to decide to abort the request as soon as possible if it does not have a chance to complete due to one of the services down the path. Could the ‘health’ of the downstream services (comprising data from their local PID controllers) be returned as headers attached to the responses and used to build a more evolved circuit breaker/early preemptive shedding mechanism? Finally, I’m curious to learn more about the previous approach because, based on the description given in this paper, it seems to be sound. When you examine the measures of goodput and latencies, there is no doubt about which one, QALM or Cinnamon, performs the best. Note that they mention a link to the QALM approach in the article. Probably should start from ;) there As always, these approaches are not for everybody. Uber’s architecture and load are kind of its own. I’m actually impatient to read the next articles in this series, specifically to learn more about the PID controller and auto-tuner. With Cinnamon we have built an efficient load shedder that uses century-old techniques to dynamically set thresholds for rejecting and estimating capacity of the services. It solves the issues we noticed with QALM (and thus any CoDel-based load shedder), namely, that Cinnamon is able to: - Quickly find a stable rejection rate - Automatically adjust the capacity of the service - Be used without setting any configuration parameters - Incur very low overhead What is interesting about this approach is that they consider all the requests to be processed to decide what to do for each new input request, as they use a (priority) queue. As mentioned, I’m curious if the mechanism could also take into account the health of all the dependent services based on the same PID measures… There are other interesting aspects in this article, such as how they measure the effect of their strategies and the comparison with the previous approach. However, it does not require more detailed notes from me than what is already presented. So, I highly encourage you to read . the original article , Found this article useful? Follow me on Linkedin , and Hackernoon ! Please 👏 this article to share it! Medium Also published here.

Unpacking Cinnamon — A New Resiliency Approach at Uber

দারুচিনি আনপ্যাক করা—উবারে একটি নতুন স্থিতিস্থাপক পদ্ধতি

Cinnamon auspacken – ein neuer Resilienz-Ansatz bei Uber

Desempacar Cinnamon: un nuevo enfoque de resiliencia en Uber

Déballer Cinnamon : une nouvelle approche de résilience chez Uber

दालचीनी को खोलना-उबेर में एक नया लचीलापन दृष्टिकोण

シナモンの開梱 — Uber における新しい回復力アプローチ

Unpacking Cinnamon - Uber의 새로운 탄력성 접근 방식

Desempacotando a Canela – Uma Nova Abordagem de Resiliência na Uber

Распаковка Cinnamon — новый подход к обеспечению устойчивости в Uber

Tarçın Paketini Açmak—Uber'de Yeni Bir Dayanıklılık Yaklaşımı

Giải mã Quế—Phương pháp tiếp cận khả năng phục hồi mới tại Uber

打开 Cinnamon 包装——Uber 的新弹性方法

PwC predicts that healthcare costs will . This increase is primarily attributed to healthcare workers experiencing burnout, the subsequent workforce shortage, disputes between payers and providers, and inflation. To ensure efficient patient care without incurring excessive operational costs, the industry is exploring innovative technologies, such as generative AI in healthcare. go up by 7% in 2024 Accenture reports that can be enhanced with AI, while a recent Forbes article suggests that this technology can spare the US medical sector . 40% of healthcare providers’ working hours at least $200 billion in annual expenses in healthcare uses machine learning algorithms to analyze unstructured data, such as patient health records, medical images, audio recordings of consultations, etc., and produce new content similar to what it has been trained on. Generative AI In this article, our generative AI development company will explain how the technology can support healthcare organizations. Generative AI use cases in healthcare Facilitating medical training and simulation Assisting in clinical diagnosis Contributing to drug development Automating administrative tasks Generating synthetic medical data Facilitating medical training and simulations can come up with realistic simulations replicating a large variety of health conditions, allowing medical students and professionals to practice in a risk-free, controlled environment. AI can generate patient models with different diseases or help simulate a surgery or another medical procedure. Generative AI in healthcare Traditional training involves pre-programmed scenarios, which are restrictive. AI, on the other hand, can quickly generate patient cases and adapt in real-time responding to the decisions the trainees make. This creates a more challenging and authentic learning experience. Real-life example The University of Michigan built a generative AI in healthcare model that can produce various scenarios for . simulating sepsis treatment The University of Pennsylvania to simulate the spread of COVID-19 and test different interventions. This helped the researchers evaluate the potential impact of social distancing and vaccination on the virus. deployed a generative AI model Assisting in clinical diagnosis Here is how generative AI for healthcare can contribute to diagnostics: . Hospitals can employ generative AI tools to enhance the traditional AI’s diagnostic abilities. This technology can convert poor-quality scans into high-resolution medical images with great details, apply , and present the results to radiologists. Generating high-quality medical images anomaly detection AI algorithms . Researchers can train generative AI models on medical images, lab tests, and other patient data to detect and diagnose early onsets of different health conditions. These algorithms can spot , , , , , and more. Additionally, AI models can unveil biomarkers that can cause particular disorders and predict disease progression. Diagnosing diseases skin cancer lung cancer hidden fractures early signs of Alzheimer’s diabetic retinopathy . Diagnosticians can turn to generative AI in healthcare if they have questions instead of looking for an answer in medical books. AI algorithms can process large amounts of data and generate answers fast, saving doctors precious time. Answering medical questions Real-life examples A team of researchers to extract and enhance features in low-quality medical scans, transforming them into high-resolution images. This approach was tested on brain MRI scans, dermoscopy, retinal fundoscopy, and cardiac ultrasound images, displaying a superior accuracy rate in anomaly detection after image enhancement. experimented with Generative Adversarial Network (GAN) models In another example, was trained on the MedQA dataset and achieved an 85% accuracy rate while answering relevant medical questions. Google admits that the algorithm still needs improvement, but it’s a strong start for generative AI as a diagnostics assistant. Google’s AI-powered Med-Palm 2 Contributing to drug development According to the Congressional Budget Office, the process of new drug development , which also includes failed drugs. Fortunately, there is evidence that AI has the potential to cut the time needed to design and screen new drugs almost by half, saving the pharma industry around in the process. Additionally, this technology can reduce costs associated with clinical trials by $28 billion per year. costs on average $1 billion to $2 billion $26 billion in annual expenses Pharmaceutical companies can deploy generative AI in healthcare to speed up drug discovery by: Designing and generating new molecules with desired properties that researchers can later evaluate in lab settings Predicting properties of novel drug candidates and proteins Generating virtual compounds with high binding affinity to the target that can be tested in computer simulations to reduce costs Forecasting side effects of novel drugs by analyzing their molecular structure You can find more information on the role of AI in drug discovery and how on our blog. it facilitates clinical trials Real-life examples The rise of strategic partnerships between biotech companies and AI startups is an early sign of generative AI taking over the pharmaceutical industry. Just recently, Recursion Pharmaceuticals for $88 million. One of them, Valence, is known for its generative AI capabilities and will work on designing drug candidates based on small and noisy datasets that are not sufficient for traditional drug discovery methods. acquired two Canadian AI startups Another interesting example comes from the University of Toronto. A research team built a generative AI system, ProteinSGM, that after studying imagery representations of existing protein structures. This tool can produce proteins at a high rate, and then another AI model, OmegaFold, is deployed to evaluate the resulting proteins’ potential. Researchers reported that most of the novel generated sequences fold into real protein structures. can generate novel realistic proteins Automating administrative tasks This is one of the most prominent generative AI use cases in healthcare. Studies show that burnout rate among physicians in the US has . Doctors suffering from this condition are more likely to be involved in incidents endangering their patients and are more inclined to alcohol abuse and having suicidal thoughts. reached a whopping 62% Fortunately, generative AI in healthcare can partially alleviate the burden off the doctors’ shoulders by streamlining administrative tasks. It can simultaneously reduce costs associated with administration, which, according to HealthAffairs, of overall healthcare spending. Here is what generative AI can do: accounts for 15%-30% Extract data from patients’ medical records and populate the corresponding health registries. Microsoft is planning to . This tool will perform various administrative tasks, such as replying to patient messages. integrate generative AI into Epic’s EHR Transcribe and summarize patient consultations, fill this information into the corresponding EHR fields, and produce clinical documentation. Microsoft’s into its clinical transcription software. Doctors can already test the beta version. Nuance integrated generative AI tech GPT-4 Generate structured health reports by analyzing patient information, such as medical history, lab results, scans, etc. Produce treatment recommendations Answer doctors’ queries Find optimal time slots for appointment scheduling based on patients’ needs and doctors’ availability Generate personalized appointment reminders and follow-up emails Review medical insurance claims and predict which ones are likely to be rejected Compose surveys to gather patient feedback on different procedures and visits, analyze it, and produce actionable insights to improve care delivery Real-life example , a medical AI startup, built a generative AI assistant that helps doctors tackle administrative duties more efficiently. This tool can access patient data, including EHRs, insurance claims, and scanned documents, give status updates, recommend care options, and answer doctors’ questions. It can even generate structured documents, such as referral letters and progress notes. Navina Navina has already scored $44 million in funding, which indicates a strong interest from the medical community. Generating synthetic medical data Medical research relies on accessing vast amounts of data on different health conditions. This data is painfully lacking, especially when it comes to rare diseases. Also, such data is expensive to gather, and its usage and sharing are governed by privacy laws. Generative AI in medicine can produce synthetic data samples that can augment real-life health datasets and are not subject to privacy regulations, as the healthcare data doesn’t belong to particular individuals. Artificial intelligence can generate EHR data, scans, etc. Real-life examples A team of German researchers built an AI-powered model, GANerAid, to . This model is based on the GAN approach and can produce medical data with the desired properties even if the original training dataset was limited in size. generate synthetic patient data for clinical trials Another team of scientists experimented with generative AI to . The researchers were motivated by restrictive data privacy regulations and the inability to effectively share patient data between hospitals. They built the EHR-M-GAN model that could derive heterogeneous, mixed-type EHR data (meaning it contains both continuous and discrete values) that realistically represents patient trajectories. synthesize electronic health records Ethical considerations and challenges of generative AI in healthcare Even though tech and consulting giants , we can also see how prominent AI experts, including Tesla CEO Elon Musk and OpenAI CEO Sam Altman, warn of the risks associated with the technology. So, which challenges does generative AI bring to healthcare? continue to invest in AI . AI models’ performance is as good as the dataset they were trained on. If the data does not fairly represent the target population, this will leave room for bias against less represented groups. As generative AI tools train on vast amounts of patient records data, they will inherit any bias present there, and it will be a challenge to detect, let alone eradicate it. Bias . Even though AI presents considerable ethical concerns, there are no official regulations yet to govern the use of this technology. The towards formalizing relevant policies, but this won’t take place in the near future. Lack of regulations US and the EU are working . AI does make mistakes, and in healthcare, the price of such mistakes is rather high. For instance, large language models (LLMs) can . Meaning they can produce syntactically probable outcomes that are factually incorrect. Healthcare organizations will need to decide when to tolerate errors and when to require the AI model to explain its conclusions. For instance, if generative AI is used to assist in cancer diagnosis, doctors are unlikely to adopt such a tool if it can’t justify its recommendations. Accuracy concerns hallucinate . Who is responsible for the final health outcome? Is it the doctor, the AI vendor, the AI developers, or yet another party? Lack of accountability can have a negative impact on motivation and performance. Accountability Ready to enhance your healthcare practice with generative AI? Generative AI algorithms are becoming increasingly powerful. Robert Pearl, a clinical professor at Stanford University School of Medicine, : said “ChatGPT is doubling in power every six months to a year. In five years, it’s going to be 30 times more powerful than it is today. In 10 years, it will be 1,000 times more powerful. What exists today is like a toy. In next-generation tools, it’s estimated there will be a trillion parameters, which is interestingly the approximate number of connections in the human brain.” AI can be a powerful ally, but if misused, it can cause significant damage. Healthcare organizations need to approach this technology with caution. If you are considering deploying AI-based solutions for healthcare, here are three tips to get you started: . Even if you decide to opt for a pre-trained, ready-made AI model, you might still want to retrain it on your proprietary dataset, which needs to be of high quality and representative of the target population. Keep medical data secure at all times and safeguard patient privacy. It would be useful to disclose which dataset an algorithm was trained on as it helps to understand where it will perform well and where it might fail. Prepare your data . Cultivate the concept of responsible AI in your organization. Make sure people know when and how to use the tools and who assumes responsibility for the final outcome. Test the generative AI models on use cases with limited impact before scaling to more sensitive applications. As mentioned earlier, generative AI can make mistakes. Decide where a small failure rate is acceptable and where you can’t afford it. For instance, 98% accuracy can suffice in administrative applications, but it’s unacceptable in diagnostics and patient-facing practices. Devise a framework that will govern the use of generative AI in healthcare at your hospital. Take control of your AI models . AI still needs human guidance, especially in the heavily regulated healthcare sector. remains an essential ingredient for the technology to succeed. The medical and administrative staff will be expected to supervise AI models, so hospitals need to focus on training people for this task. Employees, on the other hand, should be able to reinvent their daily routine, now that AI is a part of it, to use the freed-up time to produce value. Help your employees accept the technology and use it Human-in-the-loop Want to benefit from generative AI but not sure how to proceed? ! We will help you prepare your data, implement the tool, and integrate it into your operations. Drop ITRex a line Also published here.

Generative AI in Healthcare: Transforming Diagnosis, Drug Development, and More

As data scientists, we often would like to start training models as soon as possible. And that is not just because humans are impatient by nature. For example, we might want to test a very small change in someone’s project and we would like to avoid a lengthy setup phase just to test such a minor modification. That is especially apparent when working with computer vision and large image datasets. Were you ever annoyed when you had to pull a massive dataset (versioned using DVC) before training your model? Or maybe you had to pull the entire dataset just to inspect/modify a single image or a few of them? If that sounds familiar, I will show you how to solve those potential bottlenecks using DagsHub’s new data streaming client. What is the data streaming client? DagsHub’s open-source streaming client is called and essentially it allows to stream data from any DagsHub repository. In other words, our Python code will act as if all the files exist locally, but in reality, the client will dynamically download them on request. Additionally, we can also upload the data to any project hosted on DagsHub. But most importantly, using the streaming client does not require making extensive changes to our codebase. One or two additional lines of code are often enough to fully enjoy the benefits of the new client. Direct Data Access (DDA) In a nutshell, DDA offers the following: the ability to stream data in batches from any DagsHub repository, appending new data to existing datasets without pulling the entire dataset first, smart caching - files remain available after they are streamed, saving time, resources, and memory. After the high-level overview of the new client, let’s dive a bit deeper into how it actually works. Under the hood, there are two main implementations of DDA. The first one employs and it detects any calls to Python’s built-in file operations (such as , , etc.) and modifies them to retrieve the files from the DagsHub repository if they are not found locally. The biggest advantage of this approach is that most of the popular Python libraries will automatically work with this method. Python Hooks open write While this already sounds great, we should be aware of some limitations of this approach: The method will not work with frameworks that rely on I/O routines written in C/C++, for example, OpenCV. DVC commands such as and for stages that have DCV-tracked files in won’t work. dvc repro dvc run deps The second implementation is called and it is based on . Behind the scenes, the client creates a virtual mounted filesystem reflecting our DagsHub repository. That filesystem behaves like a part of our local filesystem, for example, we see all the files in the directory and we can view them. Mounted Filesystem FUSE (Filesystem in USErspace) As always, the burning question is which implementation to use. The documentation suggests that if you are working on a Windows or Mac and do not have any frameworks/libraries relying on C, use the Python Hooks approach. Otherwise, use the Mounted Filesystem. Please refer to for a comprehensive comparison of the two approaches. the documentation Use cases of the streaming client We already know what the streaming client is and how it works. Let’s now cover a few of its potential use cases to show how we can use it in our workflows. First and foremost, we can use it to reduce the time needed to start a training run. Let’s imagine we are working with an image classification problem. With the streaming client, we do not have to download the entire dataset before we actually start training a model, as Python will pretend that the files are already available and start training the models immediately. Data loaders are frequently optimized to load the next batch while the current one is being used for training. In the background, the streaming client will simply download images in batches as they are requested for training and cache them for later use. With this approach, we can potentially save a lot of time and resources. The latter can be measured by, for example, the runtime of a virtual machine. Second, we can train our models only on subsets of the entire dataset. Imagine having a dataset with images from thousands of classes, but for your particular experiment, you only want to train using a few of the selected categories. With the streaming client, your Python code will only load the data that is actually requested for training. Lastly, we can also use the streaming client to upload data to our DVC-versioned repositories. In an especially frustrating scenario, we might have tens of thousands of images in our dataset and we would like to add 10 more images that we gathered. Normally, we would have to pull all the data, add the new images and version the entire modified dataset again. With DDA, we can just directly add the 10 new images without downloading the entire dataset. A hands-on example I hope that the mentioned use cases got you excited about giving the streaming client a go! To use the benefits of the streaming client, we need to store our project on DagsHub and version the data with DVC. Please refer to to see how to set everything up. this article The goal of this tutorial is to show how quickly we can get a project up and running using the streaming client. That is why we forked an already existing project and we will show how to quickly start the training of our neural network in the cloud. For that purpose, we will use Google Colab, but you can use other cloud platforms such as AWS, etc. The project we forked is a TensorFlow implementation of CheXNet, which is a 121-layer CNN used for detecting pneumonia from chest X-rays. The original implementation is trained on the ChestX-ray14 dataset (available ), which is currently the largest publicly available chest X-ray dataset with over 100,000 images. The dataset is over 45 GB and is divided into 14 different directories, each one dedicated to a different disease. To reduce the training time of this experiment, we use a subsampled version of the dataset which is only 1GB. on Kaggle Before starting, it is worth mentioning that it is not always beneficial to use DDA. Before starting this project, I tried playing around with DDA for my . That is when I encountered an edge case in which passing a single batch during the training of the neural network was faster than downloading the batch on demand. This made the training of the first epoch much longer than downloading the entire dataset up front and only then running the training step normally. Hence, the images used for training must be large enough (in terms of their resolution/size) so that the training step of a single batch takes longer than downloading the next one. Mario vs. Wario classifier To follow along, you can use this . Notebook in Google Colab Setting up the project I chose to fork the project to my own account in order to use features such as experiment tracking with MLflow. However, you can just as well directly clone a repo and train the model without creating a fork. After opening a Notebook, we need to install some libraries: !pip install dagshub omegaconf mlflow Then, we import the required libraries: import datetime
import os
from IPython.display import display, Image The list of imports is rather short, as most of the libraries (for example, used for training the NNs) are loaded in the individual scripts. Then, we define a few constants containing values such as the owner of the repository, the name of the repository we want to clone, and the branch we want to use. keras # repo to use
REPO_OWNER= "eryk.lewinson"
REPO_NAME= "CheXNet"
REPO_BRANCH= "streaming" Having defined the required constants, we can use the handy functionality to initialize the DagsHub repository. We can then easily log the experiments with MLFlow, get access tokens, etc. We run the following snippet to set up the project in Colab: init import dagshub
dagshub.init(repo_name=REPO_NAME, repo_owner=REPO_OWNER) After running the cell, we will be prompted to authenticate in our browser. Then, we can clone the repository using the command: git clone !git clone https://{USER_NAME}:{dagshub.auth.get_token()}@dagshub.com/{REPO_OWNER}/{REPO_NAME}.git As the next step, we change the directory to the CheXNet one: %cd /content/{REPO_NAME} And then we change the branch to the one that contains the codebase we can use together with the streaming client: !git checkout {REPO_BRANCH} As we have mentioned before, the branch uses the full 45GB dataset, while the branch trains the neural network on a subsampled dataset. master streaming At this point, we are done with the setup and are ready to take the streaming client for a spin. Inspecting the dataset First, let’s explore our dataset. After cloning the codebase, we can see that the data is versioned with DVC as we can see the corresponding tracking files. However, we cannot actually inspect it from Colab. At least not yet :) Let’s use the streaming client for the task. In this tutorial, we use the Python Hooks implementation of DDA. As such, we need to run the following two lines: from dagshub.streaming import install_hooks
install_hooks() At this point, we can inspect the images using code, even though the folders are not visible in the directory tree. As we know the structure of the directories containing the images, we can use the following snippet with a relative path to print all the images in a requested directory: os.listdir("data_labeling/data/images_001/images") As you can see, this is regular Python code and the streaming client handles everything for us in the background. Executing the snippet prints the following list: ['00000417_005.png',
 '00000583_047.png',
 '00000683_002.png',
 '00000796_001.png',
 '00001088_024.png',
 '00000013_006.png',
 '00000547_009.png',
 '00000640_000.png',
 '00000732_004.png',
…] We can also display the image using the following snippet: display(Image(filename="data_labeling/data/images_001/images/00000417_005.png")) As we have mentioned before, the streaming client downloads the data on request and caches it for later use. As such, the image we have displayed was actually downloaded and we can see it in the directory tree: Now we can take the next step and start training the neural network! Training a neural network As we have already mentioned, we do not have to spend time modifying the code used for training the NN. We just need to add the already familiar two lines of code to the script. You might ask why are we doing it again when we have already executed that cell before. That is because when we are calling a script from Colab, the script runs in a separate Python context. As such, the streaming setup is no longer applied to it. train.py To make the codebase even more generic, we can use the following conditional statement combined with the setup: install_hooks if "STREAM" in sys.argv:
    from dagshub.streaming.filesystem import install_hooks
    install_hooks() is essentially an array containing the command line arguments, that is, the values that are passed while calling the script. To illustrate, we can run the training script using the following snippet: sys.argv %run modeling/src/train.py STREAM To run the script without utilizing the streaming client, we simply have to remove the argument. STREAM As soon as we run the command, we will see that the directory tree will start populating with the images that were requested for a particular batch. And that was it! By adding only two lines of code we can leverage the streaming client to start downloading the versioned data on demand. Additionally, you might want to track your experiment with MLflow. To do so using DagsHub’s setup, you also need to modify the constant in the file. MLFLOW_TRACKING_URL src/const.yaml Wrapping up In this article, we demonstrated how to use DagsHub’s streaming client to download the dataset on demand. This way, we can almost immediately start the training of our models, without first pulling all of the versioned data to our machine, be it locally or in the cloud. As you could see, the streaming client really shines for tasks such as computer vision, but it can come in handy while solving other problems as well. You can find the Notebook used in this article . The project’s codebase is available . As always, any constructive feedback is welcome. You can reach out to me on or in the comments. here here Twitter Resources https://dagshub.com/docs/feature_guide/direct_data_access/ Rajpurkar, P., Irvin, J., Zhu, K., Yang, B., Mehta, H., Duan, T., ... & Ng, A. Y. (2017). Chexnet: Radiologist-level pneumonia detection on chest x-rays with deep learning. arXiv preprint arXiv:1711.05225. https://www.kaggle.com/datasets/paultimothymooney/chest-xray-pneumonia

Optimize Model Training with a Data Streaming Client

4 Hurdles Stopping Institutional Investors From Investing in Cryptocurrency

Why do traditional asset managers and investors care about crypto assets? When they enter the market, how will they allocate across different crypto assets? How will they trade in and out? What are the potential pitfalls to be avoided? Why Crypto? Alpha or Diversification? To the extent that someone can reliably predict an asset will , they will want to own more of it. In practice, this is exceedingly difficult to do. It’s even harder to do it at scale — often with deep analysis you can make predictions better than chance ( ) but when you go to execute, your trades move the market back into fair value faster than you would like. outperform alpha, rather than beta Alpha, when you do find it, tends to be quickly exhausted. Indeed, the median asset manager doesn’t have much hope for generating significant alpha. The savvy ones opt for the next best thing: maximum diversification. And good diversification is all about correlation — you want to own assets that have low or negative correlation with each other. The best large asset managers, like , are hungry for — or, more specifically, assets that aren’t correlated with the rest of their portfolio. This is not obvious to inexperienced investors, but a negatively correlated asset can improve the performance of a portfolio from a risk/return perspective . Diversification is the only free lunch in financial markets. Bridgewater low beta assets even if that asset underperforms on average You don’t need to bet on Bitcoin appreciating much, if at all, for it to be a valuable part of your portfolio. This is crypto’s, specifically Bitcoin’s, biggest selling point to institutions: . I predict this will be the primary driver of new money flowing into Bitcoin (and perhaps a few other top coins) over the next 12–24 months. Bitcoin returns are not very correlated with other asset classes Even without a view on whether Bitcoin will replace Gold, or the Fed, or whatever: smart money is interested in its uncorrelated returns. Source: “J.P. Morgan Perspectives: Decrypting Cryptocurrencies” Of course, there are plenty of people who believe Bitcoin (and a few others) have a chance at dethroning Gold (and other precious metals) or offshore banking as a safe haven asset. Plenty of people have written about this thesis and it would indeed be a huge deal. The current market capitalization of the entire crypto economy is $250 billion whereas Gold is sitting at and offshore banking represents . In part driven by concern over central bank balance sheets, the more aggressive asset managers (like macro hedge funds) will be making this bet. $3 trillion $20 trillion There is also the opportunity to make idiosyncratic alpha bets on specific coins, chains, tokens, etc. Currently this is the realm of dedicated crypto hedge funds and VCs. Likely this will stay that way for the foreseeable future. Finally, securitized tokens. This is still a big unknown and to the extent that they become a Big Thing, the underlying platforms would likely benefit (e.g. Ethereum). How will they allocate across cryptos? Some major concerns an institutional investor may have when it comes to adding a new asset class to their portfolio: — how likely is this to just be a flash in the pan? Staying power — how can I get it and hold on to it? Custody — how likely are we to get stuck holding the bag when we need to exit? Liquidity — can we legally put this thing in our fund? any chance we get sued or otherwise screwed? Legal From this perspective, Bitcoin is likely the best bet. Especially to the extent that an institution is interested in beta, as most other liquid coins are very highly correlated to Bitcoin (with less liquidity). Source: (note: table also includes SPX = S&P 500, VIX = S&P 500 Volatility Index, GLD = Gold, TNX = 10 year Treasury Note) Sifr Data Staying Power From a perspective, Bitcoin has the longest track record and the most user support. Some institutions will still allocate along the top few coins to hedge their bets and may even place small speculative allocations on promising new projects that seem to be sufficiently different (e.g. using or new types of ). lindy effect DAGs DLT — if the asset loses most of its value, you’re still screwed. Beta is only valuable if you are pretty confident you aren’t taking on substantial negative alpha Custody Custody and liquidity, the details of how to acquire and hold a crypto asset, are still a mess. Contrary to the ethos of crypto, . They won’t be managing their own wallets. While , , and a few others have institutional offerings (or soon will), most investors will be wary to be one of the early participants. most institutions will be required to have third party custodians Coinbase Kingdom Trust It may take years of “battle testing” before these institutional-grade third party custodians are viewed as reliable and safe. Liquidity Even when you’ve figured out a custodial solution, exchange liquidity is still a real problem. While , any large trader will tell you that it’s still quite expensive to get big trades done without substantial slippage. Standing liquidity in order books is quite low, not to mention issues with moving funds and custodying coin on these exchanges. coinmarketcap may show billions a day in BTC volume Needing to move large amounts of money in/out of something like Coinbase (let alone the less reputable exchanges) is still not convenient compared to what traditional asset managers are used to. Legal Status Also, institutions are very concerned about the legal status of the assets they acquire. The biggest concern is that a bunch of these coins, and especially ICO tokens, get classified as unregistered securities sales and the issuers, exchanges, etc. are hit by a wave of lawsuits claiming fraud. Bitcoin and Ethereum have the least associated legal risk. In March of 2018, . The suggests they will view Ethereum as a commodity as well. a judge ruled that Bitcoin is a commodity recent informal SEC guidance How will they trade in and out? ETFs and CEFs Many institutions will want to hold crypto exposure through a publicly-listed vehicle with relatively low fees: an exchange-traded fund or closed-end fund (like ). One of the few available options is Grayscale’s GBTC but fees could ideally be lower (2%). A bigger problem is it trades at a very high , , though it has traded as high as a 100% premium. Grayscale’s Bitcoin Investment Trust premium to NAV about 50% as of July 2018 This indicates the tremendous demand for this sort of product but also makes it unattractive to buy into (at least for now.). An approval of an ETF would be a big help and would probably cause the NAV premium to collapse. In the same vein, the existence of on major exchanges gives a wide set of participants access to hedging and speculation on Bitcoin prices. The nice thing about enabling crypto ownership through an exchange-traded product is it opens up the playing field to a much wider set of participants: everything from a long/short equity hedge fund to an individual investor with an Interactive Brokers account. CFTC-approved futures products However, the until September and most sources I’ve spoken with put the odds of approval at 10–15%. Atlantis Asset Management chief investment strategist Michael Cohn an approval would mean “they’re putting a rubber stamp on [Bitcoin] as an asset, and I don’t think governments want to go there yet.” SEC recently postponed ruling on five bitcoin-related exchange-traded funds said Over-the-Counter For now, most institutional trading is occurring between friends, brokers, and trading desks (like and ). My guess is this is largely because institutions aren’t comfortable transferring millions of dollars to Coinbase or Gemini (let alone their less reputable competitors) — perhaps the credit lines to do so aren’t even in place yet. As the regulatory environment matures, this is likely to change and more volume will occur on exchanges. over-the-counter (OTC) Circle Cumberland As the clearing and credit components get built up, the crypto trading landscape may start to look more like the existing space of Equities or Forex trading. In those worlds, mature infrastructure exists for executing large trades. The name of the game is hiding your hand to reduce . There is a nascent world of players working on this infrastructure in the crypto space, specifically and order routers/execution algorithms. slippage dark pools Dark Pools Presumably, many of the major OTC desks are developing, or even already operating, their own dark pools. The basic premise of a dark pool is to allow large traders to execute hidden orders at prices that are to a liquid exchange’s (the “lit” venue) — at a midpoint, for example. Lit markets are probably liquid enough for this mechanism to be relatively well-behaved, although perhaps it’s still too soon even for that. There are also efforts like which aim to create dark pools. (A deep dive into how it works, and whether it has potential, will have to wait for a separate post. My initial impression is it may face the as compared to their centralized exchange brethren. This may be fatal, even in the long run.) relative Republic Protocol decentralized same hurdles and shortcoming that DEXs are experiencing What about navigating the exchanges themselves? , of exchanges. Being unregulated, these vary from fly-by-night operations to semi-mature institutions (with full KYC/AML and high performance matching engines) that may even offer colocation (I believe just Gemini right now). It’s quite difficult to “arbitrage” between many of these exchanges. Moving capital and coin takes time, and often comes with limitations and high fees. Bitcoin is traded on dozens, if not hundreds Prices are expected to differ, accounting for all kinds of factors ranging from capital controls (Korea) to risk of capital loss due to potential misrepresentation of financial condition (e.g. WEX at the moment). Source: Coinmarketcap Bitcoin Markets Order routers and execution algorithms The traditional solution is . Basically, you break up your orders into smaller pieces and spread them across multiple exchanges while accounting for: differing prices (buy low, sell high), liquidity (more is better), fees (we don’t like ‘em), and latency/likelihood of execution (high latency = bad; more uncertainty over getting filled). smart order routing There are a few players providing, or at least enabling something like, these services in crypto trading today: , , , and . SFOX Coinigy CoinRoutes TradeBlock Some of these like SFOX and CoinRoutes are custodial — they hold your capital and use their own accounts at the various exchanges to provide quick execution. (Although, they that I haven’t fully thought through to comment on). Others, like Coinigy, are just giving you API wrappers with a unified user interface, which means you need to have your own accounts at the various exchanges and move money back and forth as necessary. Finally, is a promising project looking to bring dark pool matching and order routing together in a non-custodial on-chain (with bank integration) mechanism. seem to be doing something with DEX clients OmegaOne Update: After reviewing how CoinRoutes works, I’ve realized it’s actually not custodial. It works more like a Coinigy model + consolidated order book — you’ll need to maintain accounts with the various exchanges, as well as capital there as necessary. In addition to routing your orders more intelligently, traditional players (especially in equities trading) will leverage to reduce slippage and hide their intention to make large trades. Execution algos apply more complicated logic than just placing a limit or market order. Simple examples include VWAP (volume-weighted average price) and TWAP (time-weighted average price) which execute orders to keep pace with market activity or evenly over time (respectively). buy-side execution algorithms There’s a whole world around adding pricing and trading logic to improve execution — and it really does add value, especially when making larger trades. Currently, SFOX is the main player that I can see providing algos, though I expect this to change over time. Exchange liquidity is still very low compared to what you can get OTC, and will likely remain that way until clearing and credit systems are in place. Currently, the market may simply still be immature for an ecosystem of order routers and execution algos. What are the Potential Pitfalls? There are a few potential pitfalls when it comes to executing institutional-scale crypto trades. Exchanges are risky, unregulated, and vertically integrated monoliths In the traditional financial world, trading venues handle matching and typically provide central clearing and settlement. Exchanges then face off with clearing firms or prime brokers, who then interact with brokers and traders. Contrast this with Coinbase who serves as both exchange, custodian, and broker. The crypto exchanges would benefit from taking some time to understand how the traditional trading world is designed rather than potentially assuming that a one-stop shop is the best structure for the market. These intermediaries aren’t just rent-seekers — they provide value. For example, imagine a world where clearing firms stood between traders and exchanges. Rather than the exchanges controlling all of the coin in their exchange wallet, they would rely on clearing firms to settle coin and cash balances within some period of time (e.g. within 24 hours). This allows clearing firms to extend credit to their traders across exchanges — arbitrageurs could buy BTC on Coinbase Pro and immediately sell it on Gemini and not worry about moving coin or cash. OTC trading costs are high That world is still a ways out, if it ever comes. For now, big trades will continue to occur over-the-counter. OTC is generally expensive — all-in cost of 2–5% is not unusual. In return, you can be more confident that you will get the coins (or cash, if you’re selling) quickly and securely. Coin provenance is often unknown Finally, one thing to consider is the provenance of your coins. Specifically, there is anti-money launder (AML) concern over taking possession of coins “tainted” by known criminal activity. This is a wide ranging topic that may be the elephant in the room for many crypto investors. A safe, but perhaps extreme, solution is to buy directly from miners that you can personally check for AML risk. If part of a coin you own has passed through the wallet address associated with money laundering or coin theft, you may have a hard time selling it in the future. Conclusion Many traditional asset managers want to get exposure to cryptoassets because returns are not correlated with other asset classes, thereby allowing them to improve the performance of a portfolio from a risk/return perspective (even if cryptoassets underperform). Others believe that Bitcoin (and perhaps a few others) have a chance at dethroning Gold and other precious metals as a safe haven asset. Concerns over staying power, custodianship, liquidity, and legal risk will drive traditional asset managers to concentrate investments in Bitcoin, Ethereum and perhaps a couple other coins. Trading in and out of crypto can be done through CEFs/investment trusts, and eventually ETFs, though they currently trade at significant premiums to NAV. For asset managers comfortable with custodying their own funds, OTC is the most popular option though all-in execution cost can reach 5%+. Dark pools, order routers and execution algorithms may offer better solutions in the future, though the market may still be too immature. Finally, traditional asset managers must navigate the potential pitfalls of integrated exchanges, and unknown coin provenance. Want to chat? Lately, my good friend and I have been exploring the crypto space with the intent of building a company. One remark we’ve heard over and over is how inscrutable crypto investing is, even from the perspective of a professional asset manager. It’s still too difficult for even a savvy portfolio manager to get exposure to cryptocurrency. In this article, we’ve outlined what we see as the primary reasons why. As a first step, we are exploring helping accredited investors get crypto exposure through a very low fee fund. For more information, please reach out to us at . Taylor Pearson We want to make it safe, easy, and low cost for anyone to diversify their investment portfolio with cryptocurrencies. Simple Coin Capital

Crypto investing for traditional asset managers

ICO is becoming a mainstream with the dawn of blockchain and cryptocurrency networks. More and more people are investing in ICOs which in the result is increasing the competition. In this competition planning and studying the platforms to invest in is very important. Here are some of the best ICO to invest in for 2018. 1. Assetereum is an ERC20 Token backed by real assets to give it an intrinsic value. There are 5 types of assets: Real Estate, Crypto-mining, Crypto-trading, Commercial Energy brokering, online shopping platform and Investing with social enterprise start up. Assetereum We are now sponsoring The World Seniors Snooker Championship and it will be the first time a cryptocurrency is being awarded as prize money in a major sporting event. As a result, 2 of the major players are now ambassadors for Assetereum. Buy Tokens: https://www.assetereum.com/ 2. Treon Treon (TXO) is an Ethereum blockchain-based utility token and e-wallet for paying utility bills using a robust technology in a completely decentralised and secure fashion. The utilities sector has had intermittent customer-centric innovations over the years. Disruptive blockchain technology utilised by the TXO e-wallet offers vast opportunities for utility providers to drive cost-efficiency and increase profitability while giving consumers greater control over their usage and wider financial inclusion and convenience. With Treon, we will make your utility bill payments a convenient one-click experience — anytime, anywhere. Treon (TXO) is focused on providing an exciting value-added service that will simplify the utility management and payment process like never before: consumers will effect their payments instantly using a simple consolidated dashboard. Buy Tokens: https://www.treon.io/ 3. Omega One Omega one is a new token that is going to be launched in the final phases of the year 2017 or start of 2018. Omega one is going to be the latest and cheapest way of trading cryptocurrency and tokens. It is a platform, which is going to solve the problem of illiquidity and hacking. It will provide users with a decentralized execution platform that will trade across every cryptocurrency exchange.Omega one will be the new evolution in the ICO world because it would provide increased liquidity and upgraded trade intelligence. The fees on Omega One would be reinvested in the project to make it more reliable for users. Buy Tokens: https://omega.one/ 4. Trade.io Trade.io is the latest trading and brokerage industry revolution, which allow users to profit in the process of providing services. The concept of trade.io is to provide a decentralized platform in which brokerage services can occur smoothly. It is providing its services in classic trading markets, crypto coins, and virtual products. Users will be getting 50% of all the brokerage-generated profits. Buy Tokens: https://trade.io/ 5. Kyber Network Kyber network a project led by smart security and distributed consensus algorithms specialist is a decentralized exchange that provides crypto-assets conversion. It was one of the best ICO networks in 2017 with a rising of $48 Millions USD. It took the number 10 spot in ICO statistics list. Currently, it is going through a testnet phase and plans are in progress to launch it in Q1 2018 and by early 2019.Once the cyber network is released in the market with all of its functions, it is going to take the market by storm because cyber network is going to evolve the blockchain network by providing better security and other decentralization benefits. It would allow users to make proxy payments and cross-chain payment platform. Buy Tokens: https://kyber.network/ 6. NAU NAU takes advantage of the use of mobile phones and geolocation data.NAU works on a system of connecting retailers with their customers with coupons. They are going to be the future of advertisements. They are low in cost compared to traditional advertisement. NAU has the potential of changing the advertisement world completely by giving local business access to social referral through a user affiliate program.”This is the perfect implementation of the social/local/mobile approach to connecting businesses with their customers,” explains founder and CEO Yaroslav Shakula. Buy Tokens: https://tokensale.nau.io/ 7. Giftz Giftz is a platform, which gives users access to an ability to provide a way for any business to tokenize their goods and services. This is going to be a revolution in the business world. It is a good idea when we think of how good it would be if we could use our ICO tokens and use them to buy products from different parts of the world. Giftz works on this idea, which will take the world by storm. It would eliminate the worry of taking cash to stores to buy products or thinking of having nickels to save forgetting something. If you are using ICO platform and have tokens in your wallet, you can simply use Giftz to get your desired products. Buy Tokens: https://www.giftz.io/ 8. Enigma ICO Eigma ICO was released on 11th September 2017. It is a platform, which gives everyone an ability to build his or her own crypto hedge fund. It provides users to engage in the fast-paced transaction through the system while maintaining full custody of their assets. It provides the latest research tools for quantitative traders to build, test and master investment strategies. This is a new ICO platform, which is going to provide users with new tools. It is going to sky rocket in 2018 due to its limited amount of tokens released an increase in demand due to awareness in people once it is fully functional. Buy Tokens: https://enigma.co/ 9. UnikoinGold UnikoinGold has a token designed for legal and licensed betting on esports matches and tournaments. The UnikoinGold crowdsale preregistration ended on 15th August 2017. The presale will be held from 8th to 18th September 2017 and the official crowdsale dates are yet to be announced. The crowdsale will see 20% of the 1 Billion tokens up for sale. Unikoin has previously raised $10 million from investors including Ashton Kutcher, Mark Cuban, Shari Redstone, Elisabeth Murdoch, and others.UnikoiGold has big money attached to it which is going to help in developing it to the utmost ultimate ICO in 2018. Buy Tokens: https://unikoingold.com/ 10. Nagaico Nagaico is going to change how we invest in our economic system. It is an ICO which is revolutionizing the cryptocurrencies and digital products. This ICO is going to make investment affordable to around 2 billion people. It would allow people to invest as much as they like to which means people from 3rd world countries are going to be able to buy shares in a company like Google by buying one tenth or one-hundredth of one Google share through CFD contract. Naga will handle all the clearance of trades, which would allow small investors to get access to the world’s most influential and largest financial markets. Buy Tokens: https://www.thenagacoin.com/ For More Info: https://www.assetereum.com/ Source: https://steemit.com/cryptocurrency/@fatos/top-10-ico-s-for-investment-in-2018

Stories, News & Mentions about OMEGAPRO

Omegapro’s Stories on HackerNoon

Omegapro's News & Mentions