INZINT

inzint

Understanding Parallel Programming: Thread Management for Beginners

Integral Neural Networks (INNs) are flexible architectures that trained once can be transformed to arbitrary user-defined size without any fine-tuning. As sound waves (music) can be sampled at any desired sampling rate (sound quality) INNs can change data and parameters shape (DNN quality) dynamically. The team of TheStage.ai presented their paper, “Integral Neural Networks,” at the IEEE/CVF CVPR 2023 conference. This work was recognized as one of only 12 ‘Award Candidate’ papers featured at the event. INNs are a new class of neural networks that combine continuous parameters and integral operators to represent basic layers. During the inference stage, INNs convert to a vanilla DNN representation by discrete sampling from continuous weights. Parameters of such networks are continuous along filter and channel dimensions which leads to structured pruning without fine-tuning just by re-discretization along those dimensions. In this article, we will guide you through the process of transforming a 4x image super-resolution EDSR model to an INN, then show how to achieve structured pruning of the model. Finally, we will convert the INN back to a discrete DNN and deploy it on an Nvidia GPU for efficient inference. This article will proceed as follows: A brief introduction to INNs.
Overview of EDSR network for super-resolution task.
TorchIntegral framework application to obtain integral EDSR in a single line of code.
Structural pruning of INN without INN fine-tuning (fast pipeline).
Deploy pruned models on Nvidia GPUs. For further information and updates, please check the following resources: INN project site INN project Github Support code of this article INNs for DNNs pruning without fine-tuning Layers in INNs are replaced by integral operators, but the practical evaluation of integral operators requires the discretization of input signals for the usage of numerical integration methods. It turns out that the layers in INNs are designed in such a way as to coincide with classical DNN layers (fully-connected, convolutions) after discretization. EDSR 4x Image Super-Resolution pruning Image super-resolution is a well-known computer vision task where an image should be enhanced with a known or unknown degradation operator. We consider classical super-resolution form using bicubic downsampling as a degradation operator. Many architectures can be used for image super-resolution tasks, including high-end neural networks based on diffusion models and transformers. In this document, we will focus on the 4x EDSR architecture. EDSR architecture is well-suited for our demonstration as it comprises ResNet (which is widely used in many deep learning problems) and a 4x upsampling block at the end. A schematic description of EDSR can be found in the following figures. Specifics of EDSR architecture pruning Structured pruning, which involves deleting entire filters or channels, has unique implications for residual blocks, which serve as the primary building blocks in EDSR. Since each state is updated by adding a Conv -> ReLU -> Conv block to the input, the input and output signals must have the same number of parameters. This can be efficiently managed in TorchIntegral by creating a pruning dependency graph. The figure below illustrates that the second convolution of each residual block forms a single group. To prune the second convolution in a residual block, it is necessary to prune all second convolutions in each residual block. However, for a more flexible setup, we prune filters of the first convolutions, and therefore channels of the second convolutions, in all residual blocks. Conversion of EDSR model to INN EDSR For conversions from the pre-trained DNNs, we utilize our special filter-channels permutation algorithm with further smooth interpolation. The permutation algorithm preserves model quality while making the weights of the DNNs look like sampled from continuous functions. import torch
import torchintegral as inn
from super_image import EdsrModel

# creating 4x EDSR model
model = EdsrModel.from_pretrained("eugenesiow/edsr", scale=4).cuda()

# Transform model layers to integral.
# continous_dims and discrete dims define which dimensions
# of parameters tensors should be parametrized continuously
# or stay fixed size as in discrete networks.
# In our case we make all filter and channel dimensions
# to be continuous excluding convolutions of the upsample block.
model = inn.IntegralWrapper(init_from_discrete=True)(
    model, example_input, continuous_dims, discrete_dims
).cuda() Integration grid tuning: structured post-training pruning of DNNs Integration grid tuning is the operation that selects smoothly (under SGD optimization) parameter tensors whose filters should be sampled for user-defined numbers. Unlike the filter/channel deletion methods, INNs generate filters that can combine several discrete filters because of interpolation operation. INNs introduce soft select-by-index operation on the tensor of parameters along filter and channel dimensions. # Set trainable gird for each integral layer
# Each group should have the same grid
# During the sum of continuous signals
# We need to sample it using the same set of points

for group in model.groups:
    new_size = 224 if 'operator' in group.operations else 128 
    group.reset_grid(inn.TrainableGrid1D(new_size))

# Prepare model for tuning of integration grid
model.grid_tuning()
# Start training
train(model, train_data, test_data) Integration grid tuning is a fast optimization process that can be carried out on a small calibration set. The result of this optimization is a structurally compressed DNN. Tests on a single Nvidia A4000 show that integration grid tuning on the full Div2k dataset requires 4 minutes. A distributed setup on 4x A4000 demonstrates almost a 4x speedup, resulting in an optimization time of just 1 minute. During our experiments, we found that 500 images give the same result as a full train set Div2k of 4000 images. Performance The resulting INN model can be easily converted to a discrete model and deployed on any NVIDIA GPU. We provide frames per second (FPS) on the RTX A4000 with an input resolution of 64x64. The compressed model achieves almost a 2x speedup. To convert the pruned INN model to a discrete model the following line of code can be used: model = model.transform_to_discrete()
# then model can be compiled, for instance
# compilation can add an additional 1.4x speedup for inference
model = torch.compile(model, backend='cudagraphs') Model

Size FP16

FPS RTX A4000

PSNR



EDSR orig.

75 MB

170

30.65



INN EDSR 30%

52 MB

230

30.43



INN EDSR 40%

45 MB

270

30.34



INN EDSR 50%

37 MB

320

30.25 Conclusion In this article, we have presented an overview of the CVPR2023 award candidate paper, “Integral Neural Networks”. It was applied for the post-training pruning of the 4x EDSR model, achieving nearly a 2x speedup with a single line of code and 1-minute fine-tuning of the integration grid. In our future articles, we will present more applications of the INNs and will cover more details on efficient model deployment. Stay tuned: INN project site INN project Github Support code of this article Thank you for your attention! Integral Neural Networks (INNs) are flexible architectures that trained once can be transformed to arbitrary user-defined size without any fine-tuning. As sound waves (music) can be sampled at any desired sampling rate (sound quality) INNs can change data and parameters shape (DNN quality) dynamically. Integral Neural Networks (INNs) are flexible architectures that trained once can be transformed to arbitrary user-defined size without any fine-tuning. As sound waves (music) can be sampled at any desired sampling rate (sound quality) INNs can change data and parameters shape (DNN quality) dynamically. The team of TheStage.ai presented their paper, “Integral Neural Networks,” at the IEEE/CVF CVPR 2023 conference. This work was recognized as one of only 12 ‘Award Candidate’ papers featured at the event. INNs are a new class of neural networks that combine continuous parameters and integral operators to represent basic layers. During the inference stage, INNs convert to a vanilla DNN representation by discrete sampling from continuous weights. Parameters of such networks are continuous along filter and channel dimensions which leads to structured pruning without fine-tuning just by re-discretization along those dimensions. TheStage.ai In this article, we will guide you through the process of transforming a 4x image super-resolution EDSR model to an INN, then show how to achieve structured pruning of the model. Finally, we will convert the INN back to a discrete DNN and deploy it on an Nvidia GPU for efficient inference. This article will proceed as follows: A brief introduction to INNs. Overview of EDSR network for super-resolution task. TorchIntegral framework application to obtain integral EDSR in a single line of code. Structural pruning of INN without INN fine-tuning (fast pipeline). Deploy pruned models on Nvidia GPUs. A brief introduction to INNs. A brief introduction to INNs. Overview of EDSR network for super-resolution task. Overview of EDSR network for super-resolution task. TorchIntegral framework application to obtain integral EDSR in a single line of code. TorchIntegral framework application to obtain integral EDSR in a single line of code. Structural pruning of INN without INN fine-tuning (fast pipeline). Structural pruning of INN without INN fine-tuning (fast pipeline). Deploy pruned models on Nvidia GPUs. Deploy pruned models on Nvidia GPUs. For further information and updates, please check the following resources: INN project site INN project site INN project site INN project Github INN project Github INN project Github Support code of this article Support code of this article Support code of this article INNs for DNNs pruning without fine-tuning Layers in INNs are replaced by integral operators, but the practical evaluation of integral operators requires the discretization of input signals for the usage of numerical integration methods. It turns out that the layers in INNs are designed in such a way as to coincide with classical DNN layers (fully-connected, convolutions) after discretization. EDSR 4x Image Super-Resolution pruning Image super-resolution is a well-known computer vision task where an image should be enhanced with a known or unknown degradation operator. We consider classical super-resolution form using bicubic downsampling as a degradation operator. Many architectures can be used for image super-resolution tasks, including high-end neural networks based on diffusion models and transformers. In this document, we will focus on the 4x EDSR architecture. EDSR architecture is well-suited for our demonstration as it comprises ResNet (which is widely used in many deep learning problems) and a 4x upsampling block at the end. A schematic description of EDSR can be found in the following figures. Specifics of EDSR architecture pruning Structured pruning, which involves deleting entire filters or channels, has unique implications for residual blocks, which serve as the primary building blocks in EDSR. Since each state is updated by adding a Conv -> ReLU -> Conv block to the input, the input and output signals must have the same number of parameters. This can be efficiently managed in TorchIntegral by creating a pruning dependency graph . The figure below illustrates that the second convolution of each residual block forms a single group. pruning dependency graph To prune the second convolution in a residual block, it is necessary to prune all second convolutions in each residual block. However, for a more flexible setup, we prune filters of the first convolutions, and therefore channels of the second convolutions, in all residual blocks. Conversion of EDSR model to INN EDSR For conversions from the pre-trained DNNs, we utilize our special filter-channels permutation algorithm with further smooth interpolation. The permutation algorithm preserves model quality while making the weights of the DNNs look like sampled from continuous functions. import torch
import torchintegral as inn
from super_image import EdsrModel

# creating 4x EDSR model
model = EdsrModel.from_pretrained("eugenesiow/edsr", scale=4).cuda()

# Transform model layers to integral.
# continous_dims and discrete dims define which dimensions
# of parameters tensors should be parametrized continuously
# or stay fixed size as in discrete networks.
# In our case we make all filter and channel dimensions
# to be continuous excluding convolutions of the upsample block.
model = inn.IntegralWrapper(init_from_discrete=True)(
    model, example_input, continuous_dims, discrete_dims
).cuda() import torch
import torchintegral as inn
from super_image import EdsrModel

# creating 4x EDSR model
model = EdsrModel.from_pretrained("eugenesiow/edsr", scale=4).cuda()

# Transform model layers to integral.
# continous_dims and discrete dims define which dimensions
# of parameters tensors should be parametrized continuously
# or stay fixed size as in discrete networks.
# In our case we make all filter and channel dimensions
# to be continuous excluding convolutions of the upsample block.
model = inn.IntegralWrapper(init_from_discrete=True)(
    model, example_input, continuous_dims, discrete_dims
).cuda() Integration grid tuning: structured post-training pruning of DNNs Integration grid tuning is the operation that selects smoothly (under SGD optimization) parameter tensors whose filters should be sampled for user-defined numbers. Unlike the filter/channel deletion methods, INNs generate filters that can combine several discrete filters because of interpolation operation. INNs introduce soft select-by-index operation on the tensor of parameters along filter and channel dimensions. INNs introduce soft select-by-index operation on the tensor of parameters along filter and channel dimensions. soft select-by-index operation # Set trainable gird for each integral layer
# Each group should have the same grid
# During the sum of continuous signals
# We need to sample it using the same set of points

for group in model.groups:
    new_size = 224 if 'operator' in group.operations else 128 
    group.reset_grid(inn.TrainableGrid1D(new_size))

# Prepare model for tuning of integration grid
model.grid_tuning()
# Start training
train(model, train_data, test_data) # Set trainable gird for each integral layer
# Each group should have the same grid
# During the sum of continuous signals
# We need to sample it using the same set of points

for group in model.groups:
    new_size = 224 if 'operator' in group.operations else 128 
    group.reset_grid(inn.TrainableGrid1D(new_size))

# Prepare model for tuning of integration grid
model.grid_tuning()
# Start training
train(model, train_data, test_data) Integration grid tuning is a fast optimization process that can be carried out on a small calibration set. The result of this optimization is a structurally compressed DNN. Tests on a single Nvidia A4000 show that integration grid tuning on the full Div2k dataset requires 4 minutes. A distributed setup on 4x A4000 demonstrates almost a 4x speedup, resulting in an optimization time of just 1 minute. Integration grid tuning is a fast optimization process that can be carried out on a small calibration set. The result of this optimization is a structurally compressed DNN. Tests on a single Nvidia A4000 show that integration grid tuning on the full Div2k dataset requires 4 minutes . A distributed setup on 4x A4000 demonstrates almost a 4x speedup, resulting in an optimization time of just 1 minute . fast optimization process requires 4 minutes just 1 minute During our experiments, we found that 500 images give the same result as a full train set Div2k of 4000 images. Performance The resulting INN model can be easily converted to a discrete model and deployed on any NVIDIA GPU. We provide frames per second (FPS) on the RTX A4000 with an input resolution of 64x64. The compressed model achieves almost a 2x speedup. To convert the pruned INN model to a discrete model the following line of code can be used: model = model.transform_to_discrete()
# then model can be compiled, for instance
# compilation can add an additional 1.4x speedup for inference
model = torch.compile(model, backend='cudagraphs') model = model.transform_to_discrete()
# then model can be compiled, for instance
# compilation can add an additional 1.4x speedup for inference
model = torch.compile(model, backend='cudagraphs') Model

Size FP16

FPS RTX A4000

PSNR



EDSR orig.

75 MB

170

30.65



INN EDSR 30%

52 MB

230

30.43



INN EDSR 40%

45 MB

270

30.34



INN EDSR 50%

37 MB

320

30.25 Model

Size FP16

FPS RTX A4000

PSNR EDSR orig.

75 MB

170

30.65 INN EDSR 30%

52 MB

230

30.43 INN EDSR 40%

45 MB

270

30.34 INN EDSR 50%

37 MB

320

30.25 Model Size FP16 FPS RTX A4000 PSNR Model Model Size FP16 Size FP16 FPS RTX A4000 FPS RTX A4000 PSNR PSNR EDSR orig. 75 MB 170 30.65 EDSR orig. EDSR orig. 75 MB 75 MB 170 170 30.65 30.65 INN EDSR 30% 52 MB 230 30.43 INN EDSR 30% INN EDSR 30% 52 MB 52 MB 230 230 30.43 30.43 INN EDSR 40% 45 MB 270 30.34 INN EDSR 40% INN EDSR 40% 45 MB 45 MB 270 270 30.34 30.34 INN EDSR 50% 37 MB 320 30.25 INN EDSR 50% INN EDSR 50% 37 MB 37 MB 320 320 30.25 30.25 Conclusion In this article, we have presented an overview of the CVPR2023 award candidate paper, “Integral Neural Networks”. It was applied for the post-training pruning of the 4x EDSR model, achieving nearly a 2x speedup with a single line of code and 1-minute fine-tuning of the integration grid. In our future articles, we will present more applications of the INNs and will cover more details on efficient model deployment. Stay tuned: INN project site INN project site INN project site INN project Github INN project Github INN project Github Support code of this article Support code of this article Support code of this article Thank you for your attention!

Unleashing 2x Acceleration for DNNs: Transforming Models with Integral Neural Networks in Just 1 Min

DNN-এর জন্য 2x ত্বরণ আনলিশ করা: মাত্র 1 মিনিটে ইন্টিগ্রাল নিউরাল নেটওয়ার্ক সহ মডেলগুলিকে রূপান্তর করা

Entfesseln Sie die doppelte Beschleunigung für DNNs: Transformieren Sie Modelle mit integralen neuronalen Netzen in nur 1 Minute

Liberando una aceleración 2x para DNN: transformando modelos con redes neuronales integrales en solo 1 minuto

Libérer une accélération 2x pour les DNN : transformer des modèles avec des réseaux de neurones intégraux en seulement 1 minute

डीएनएन के लिए 2x एक्सेलेरेशन जारी करना: केवल 1 मिनट में इंटीग्रल न्यूरल नेटवर्क के साथ मॉडल को बदलना

DNN の 2 倍の高速化を解き放つ: 統合ニューラル ネットワークを使用してわずか 1 分でモデルを変換

DNN을 위한 2배 가속 실현: 단 1분 만에 통합 신경망을 사용하여 모델 변환

Liberando aceleração 2x para DNNs: transformando modelos com redes neurais integrais em apenas 1 minuto

DNN'ler için 2 Kat Hızlanmayı Ortaya Çıkarma: Sadece 1 Dakikada Entegre Sinir Ağlarıyla Modelleri Dönüştürme

Giải phóng khả năng tăng tốc gấp 2 lần cho DNN: Chuyển đổi mô hình với Mạng thần kinh tích hợp chỉ trong 1 phút

为 DNN 释放 2 倍加速：只需 1 分钟即可使用集成神经网络转换模型

In this article, I will tell you how to write a telegram bot that integrates with the real world via a microcontroller (Arduino). Everything is remembered better in practice. So, I will explain everything through a practical project from my experience. The project code is published on my GitHub . This article will explain how to create a telegram bot, process messages, and respond to users. Also, in the end, will be given some ideas for future projects for you. ‘Smart Queue’ project page Shortly about project I did a project for controlling the queue to our office gaming room by Arduino-based Telegram. The Arduino with servo motor was installed to lock and unlock the door. People could take a queue and check how many people were in the queue via telegram bot. In this project, NodeMCU has been used but the same logic works for Arduino with ESP32/ESP8266. Telegram Telegram is a messenger with mobile applications (iPhone and Android) and desktop applications (Mac, Windows, and Linux). In Telegram, we have an opportunity to create a bot. We will control it with our microcontroller by processing messages from users. Creating Telegram Bot To create a Telegram Bot, we need to request it from the “botfather” telegram bot. All you need is to find the bot by search or get it by the link: . The bot is user-friendly and as you open the chat with the bot, our next steps should be clear for you. https://t.me/botfather Send /start command to start the bot. Send /newbot command to create a new bot. We will be asked to enter the name and username of the new bot. After the successful creation of the bot, the botfather will send us a token. This one will be required to receive all messages sent to the bot and send messages from the bot to users. Connection We will use a servo motor to lock and unlock the door and an LED to indicate if the door is locked. Servo motors have 3 pins: orange, red and brown: Red - 5V, Brown - GND, Orange - 14. LED connected to pin 2. Code First of all, we should import all required libraries: #ifdef ESP32
  #include <WiFi.h>
#else
  #include <ESP8266WiFi.h>
#endif
#include <WiFiClientSecure.h>
#include <UniversalTelegramBot.h>  
#include <ArduinoJson.h>

#include <Servo.h>
Servo myservo; Write credentials of wi-fi to which microcontroller will be connected: const char* ssid = "";      //wifi name
const char* password = "";  //wifi password Write token of our bot, which we received after its creation: // Initialize Telegram BOT
#define BOTtoken "XXXXXXXXX:XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX"  // your Bot Token (Get from Botfather) Define the outputs and other required variables: // Checks for new messages every 0.1 second.
int botRequestDelay = 100;
unsigned long lastTimeBotRan;

int servo_pin = 14;//servo pin
int servo_locked_value = 180;
int servo_unlocked_value = 30;

const int ledPin = 2;

String queue[10] ={};//array of queue
int queue_array_len = 0;//len of queue
bool room_is_free = true;//initially the room is free
bool door_is_locked = true;// the door is locked Create a new wi-fi client and a bot with the token and the client: WiFiClientSecure client;
UniversalTelegramBot bot(BOTtoken, client); In the setup section we will initialize led and servo, begin serial and connect to wi-fi: void setup() {
  Serial.begin(115200);

  #ifdef ESP8266
    configTime(0, 0, "pool.ntp.org");      // get UTC time via NTP
    client.setTrustAnchors(&cert); // Add root certificate for api.telegram.org
  #endif

  pinMode(ledPin, OUTPUT);
  digitalWrite(ledPin, HIGH);
  
  // Connect to Wi-Fi
  WiFi.mode(WIFI_STA);
  WiFi.begin(ssid, password);
  #ifdef ESP32
    client.setCACert(TELEGRAM_CERTIFICATE_ROOT); // Add root certificate for api.telegram.org
  #endif
  while (WiFi.status() != WL_CONNECTED) {
    delay(1000);
    Serial.println("Connecting to WiFi..");
  }
  // Print ESP32 Local IP Address
  Serial.println(WiFi.localIP());

  //Servo - close:
  myservo.attach(servo_pin);
  myservo.write(180);
} In the loop section we will do three main things: check if there are any new messages to the telegram bot; check if the room is free and if so, invite the next person in the queue; indicate if door locked and open/close the door void loop() {
  if (millis() > lastTimeBotRan + botRequestDelay)  {
    int numNewMessages = bot.getUpdates(bot.last_message_received + 1);

    while(numNewMessages) {
      Serial.println("got response");
      handleNewMessages(numNewMessages);
      numNewMessages = bot.getUpdates(bot.last_message_received + 1);
    }
    lastTimeBotRan = millis();

    //check if the room free
    if(room_is_free ==true && queue[0]!=""){
      room_is_free = false;
      //invite to the room:
      Serial.println("Invinting to the room: "+queue[0]);
      bot.sendMessage(queue[0], "The room is free! \n Please come and send /unlock_the_door_from_outside to unlock the door.", "");
    }

    //Indicating and opening/closing the door:
    digitalWrite(ledPin, door_is_locked);//ledState
    if(door_is_locked){
      myservo.write(servo_locked_value);//locked
    }else{
      myservo.write(servo_unlocked_value);//unlocked
    }
    
  }
} Here is how we will handle new messages and respond to /start message: void handleNewMessages(int numNewMessages) {
  Serial.println("handleNewMessages");
  Serial.println(String(numNewMessages));

  for (int i=0; i<numNewMessages; i++) {
    // Chat id of the requester
    String chat_id = String(bot.messages[i].chat_id);

    // Print the received message
    String text = bot.messages[i].text;
    Serial.print("From: ");Serial.println(chat_id);
    Serial.print("Message: ");Serial.println(text);

    String from_name = bot.messages[i].from_name;

    if (text == "/start") {
      String welcome = "Welcome, " + from_name + ".\n";
      welcome += "By this bot you can stand in line to the Gaming room.\n";
      welcome += "Please see my /options.\n\n";
      bot.sendMessage(chat_id, welcome, "");
    }    
  }
} Here is how we respond to /options message and send buttons for easy interactions: if (text == "/options") {
  String keyboardJson = "[[\"/Stand_in_line\"],[\"/cancel\", \"/status\"]]";
  bot.sendMessageWithReplyKeyboard(chat_id, "Here is my options:", "", keyboardJson, true);
} When we receive /Stand_in_line message we need to check if person already in the queue and if not add the person to the queue and inform it: if (text == "/Stand_in_line") {
  //check if person already in the queue:
  if(if_chat_id_not_in_queue(chat_id)){
    //add person to the queue
    queue[queue_array_len] = chat_id;//array of queue
    queue_array_len++;
    bot.sendMessage(chat_id, "You joined the queue.", "");
  }else{
    bot.sendMessage(chat_id, "You are already in the queue.", "");
  }
      
  //printing the queue
  print_queue();
} ** ** When we receive /cancel message, we need to remove the person from the queue by checking if he is there. Also if the person was already invited to the room, we will set the room status back to free. if (text == "/cancel") {
  //check if person already in the queue:
  if(if_chat_id_not_in_queue(chat_id)){
    bot.sendMessage(chat_id, "You were not in the queue.", "");
  }else{
    //checking if the person first one to set room to free again.
    if (queue[0]=chat_id){
      room_is_free = true;
    }   
    //remove person from the queue
    for(int i = 0; i < queue_array_len; i++){
      if(queue[i]=chat_id){
        queue[i] = "";
        //move items after the deleted one:
        for(int j = i;j < queue_array_len; j++){
          queue[j] = queue[j+1];
        }
      }
    }
    queue_array_len--;
    bot.sendMessage(chat_id, "You are no longer in the queue.", "");

    //printing the queue
    print_queue();
  }
} When we receive /status message, we will send how many persons in the queue and if person in it, we will also send person's position in the queue: if (text == "/status") {
  //printing the queue
  print_queue();
      
  int current_person_id_in_queue = -1;
  for(int i = 0; i < queue_array_len; i++){
    if(queue[i]==chat_id){
      current_person_id_in_queue = i+1;
    }
  }
  String message = "The queue consist of " + String(queue_array_len ) + " person(s).\n";
  if(current_person_id_in_queue != -1){
        message +="You are #"+String(current_person_id_in_queue)+ " in line.\n";
  }
      
  bot.sendMessage(chat_id, message, "");
} When person queue comes, we will send an invitation message: When we receive /unlock_the_door_from_outside message we need to check if the person is first in the queue and if so, unlock the door and send response: if (text == "/unlock_the_door_from_outside") {
  //checking if this first person in queue:
  if(chat_id == queue[0]){
    door_is_locked = false;
    myservo.write(servo_unlocked_value);//unlocked
    bot.sendMessage(chat_id, "The door is unlocked! \nCome in and lock the door by sending /lock_the_door_from_inside.", "");
  }else{
    bot.sendMessage(chat_id, "Something went wrong. \nPlease try to check your status in queue by /status command.", "");
  }
} In the next /lock_the_door_from_inside and /unlock_the_door_from_inside commands as before we need to check if the person is the first one in the queue. We do this to make sure the door is locked or unlocked by the right person. For /lock_the_door_from_inside message, we need to just lock the door: if (text == "/lock_the_door_from_inside") {
  //checking if this first person in queue:
  if(chat_id == queue[0]){
    door_is_locked = true;
    myservo.write(servo_locked_value);//locked
    bot.sendMessage(chat_id, "The door is locked! \nTo unlock the door send /unlock_the_door_from_inside.", "");
  }else{
    bot.sendMessage(chat_id, "Something went wrong. \nPlease try to check your status in queue by /status command.", "");
  }
} As we receive /unlock_the_door_from_inside message, we will unlock the door for three seconds and lock it back. Also, we will remove the person from the queue and set the room status back to free. This required to invite next person from the queue: if (text == "/unlock_the_door_from_inside") {
    //checking if this first person in queue:
    if(chat_id == queue[0]){
    door_is_locked = false;
    myservo.write(servo_unlocked_value);//unlocked
    bot.sendMessage(chat_id, "The door is unlocked! We will lock it in 3 sec", "");
    delay(3000);
    door_is_locked = true;
    room_is_free = true;
    myservo.write(servo_locked_value);//locked
        
    //remove the person from the queue
    for(int i = 0; i < queue_array_len; i++){
        if(queue[i]=chat_id){
        queue[i] = "";
        //move items after the deleted one:
        for(int j = i;j < queue_array_len; j++){
            queue[j] = queue[j+1];
        }
        }
    }
    queue_array_len--;
    bot.sendMessage(chat_id, "The door is locked. Bye!", "");

    }else{
    bot.sendMessage(chat_id, "Something went wrong. \nPlease try to check your status in queue by /status command.", "");
    }
} Here are our helper functions. These two parts of the code are used many times. So, I decided to separate them for convenience of the code: //check if person already in the queue:
bool if_chat_id_not_in_queue(String chat_id){
  for(int i = 0; i < queue_array_len; i++){ 
    if(chat_id == queue[i]){
      return false;
    }
  }
  return true;
}

//printing the queue
void print_queue(){
  Serial.print("In queue - ");Serial.println(queue_array_len);
  for(int i = 0; i < queue_array_len; i++){
    Serial.println(queue[i]);
  }
} Conclusion Through this tutorial you’ve learned how to interact with your microcontroller with Telegram Bot. The microcontroller can process messages sent to the bot and send messages back. This project is an example and showcase for you. The possibilities are limited by your imagination. Here are some ideas for you: make a Motion Detector which will send a message as motion will be detected; connect a camera module and send image by request; control LED strip or get any sensor indication. Also, there could be done such as making a big led matrix and giving control of it to everyone by telegram bot. global project The great thing about integrated Telegram bot projects is that they can be controlled from anywhere in the world. This gives your projects the opportunity to reach a new level.

Integrate Your Telegram Bot With Real World via ESP32/ESP8266 Arduino Microcontroller

Integre su bot de Telegram con el mundo real a través del microcontrolador Arduino ESP32/ESP8266

[Tutorial] Build a Gender Classifier for Live Webcam Stream using Tensorflow and OpenCV

https://cdn-images-1.medium.com/max/2000/1*bhonbY4jHhFNGXzBkodmbw.png In my I talked about how you can quickly create a Siri Shortcut using NSUserActivity. NSUserActivity is great for scenarios where you want to open your app through . In more advanced scenario, where you want finer control over the Shortcuts experience it is recommended that you use . last post Shortcuts Intents In this post I will show you how you can get started with Intents and how you can setup intents definition file which serves as the blueprint for your Siri Shortcuts. What are Intents? Intents allows you to have finer control over Shortcuts. This means instead of Siri saying “ ” Your coffee has been ordered and it will arrive at a location Siri can say something more meaningful like “ ” Your cappuccino has been ordered and it will arrive at your office in 10 minutes Intents can also be used to create custom responses and even a custom user interface which will allow users to place their order without even opening the app. Don’t worry too much about custom user interface for intents for now, I will cover that in the next post. Defining your Intents: Intents are defined in an Intents definition file. Add a new file to your project and search for “intents” in the search bar. Select the “ ” as shown below: SiriKit Intent Definition File By default the Intents file is empty. Click on the “+” sign in the bottom which adds a new intent. Rename the intent to “ ” from its default name. OrderCoffee Intents definition file is divided into multiple sections. Let’s start with “ ” section at the top. This is where you define the name of your intent, description, category and the default image. Custom Intent Since, our app is responsible for ordering coffee that is why we selected the category as “ ”. The Confirmation checkmark is checked indicating that user confirmation is required in order to execute this intent. Order Next, we will setup the parameters for your intent. Parameters is the additional data you can pass to the intents. Parameters are also used by Siri when used with Siri Voice Shortcuts. We are using two parameters and . One important thing to notice is that the deliveryLocation is of type Location and not String. This means we can even use it to calculate the estimated time of arrival. coffeeName deliveryLocation You can also use your custom classes as Parameters. In those cases the Parameter type will be custom. Finally, we will set “ ”. Shortcut Types allows Siri to learn more about your Shortcut behaviors. Shortcut Types really depends on the app and should be defined in a way which makes sense. In our app we are defining two shortcut types so Siri can learn the behavior when coffee and location is available and also when the location is not available. Shortcut Types Go ahead and build your application. Behind the scenes Xcode will look at the Intents file and create a corresponding Swift class “ ”. In the next section we will learn how to donate our intents. OrderCoffeeIntent Donating Intents Finally, the time has come to donate our intents. We start by creating a placemark instance which represents, where the coffee needs to be delivered. This can be done by using the CLLocation and CLPlacemark classes as shown below: Next, we will create an instance of class, which was automatically generated by the Intents Definition File. OrderCoffeeIntent Finally, we will donate our intent using the INInteraction instance as shown below: That’s it! Next run the app and donate your intent. After you have donated the shortcut you should be able to see your donations under “ ” as shown below. Siri App Suggestions Nice! In the next post we will take our app to the next level by implementing a custom user interface for your shortcut. This will allow us to place an order for coffee without even opening the app. Mastering Siri Shortcuts for iOS If you have enjoyed this post and want to learn more about Siri Shortcuts then check out my 3+ hour Udemy course “ ”. The course goes into much more detail in how to create Siri Shortcuts using NSUserActivity, Intents, Custom Responses, Custom User Interface and much more. Mastering Siri Shortcuts in iOS _Streamline the things you do often with shortcuts_www.udemy.com Mastering Siri Shortcuts in iOS | Udemy

Getting Started with Siri Shortcuts in iOS 12 Using Intents

The title is a lie. This concept, however, is a in . Go ahead and grab a cup of Java, before we dive into the concept of primitive data types. primitive concept Java Boolean, char, byte, short, int, long, float, and double are all primitive data types in Java. These are all words that Java knows. These values are stored in the , as opposed to the . Variables that are on the stack are accessible directly from memory, and can run very fast. Objects are on the heap, and take more time to access. stack heap Every primitive type in Java has a wrapper class. has short Short has int Integer has long Long has boolean Boolean has char Character has float Float has double Double has byte Byte The wrappers have methods attached to them that the primitive types do not have. These include… Image from tutorialspoint.com Image from tutorialspoint.com Image from tutorialspoint.com Wrapper classes are primitives are also stored differently in memory. The wrapper classes are stored on the as a reference to an object on the . stack heap class PrimitivePost{public static void main(String args[]){//declearing a boolean, this returns true or falseboolean t = true;// declaring characterchar a = ‘G’;// declaring bytebyte b = 4;// declaring shortshort s = 56; // declaring intint i=89;//declaring a longlong l = 244843984;// declaring a float — for float use ‘f’ as suffixfloat f = 4.7333434f; // declaring a double — default fraction value is double in javadouble d = 4.355453532; System.out.println(“boolean: “ + t);System.out.println(“char: “ + a);System.out.println(“byte: “ + b);System.out.println(“short: “ + s);System.out.println(“int: “ + i);System.out.println(“long: “ + l);System.out.println(“float: “ + f);System.out.println(“double: “ + d);}}

A Concept More Primitive Than The Cavemen

Exception handling is annoying. It would be completely unnecessary if everything in the world worked the way it’s supposed to. But of course that would be a fantasy. Haskell can’t change reality. But its error facilities are a lot better than most languages. This week we’ll look at some common error handling patterns. We’ll see a couple clear instances where Haskell has simpler and cleaner code. Using Nothing The most basic example we have in Haskell is the type. This allows us to encapsulate any computation at all with the possibility of failure. Why is this better than similar ideas in other languages? Well let’s take Java for example. It’s easy enough to encapsulate when you’re dealing with pointer types. You can use the “null” pointer to be your failure case. Maybe Maybe public MyObject squareRoot(int x) {  if (x < 0) {    return nil;  } else {    return MyObject(Math.sqrt(x));  }} But this has a few disadvantages. First of all, null pointers (in general) look the same as regular pointers to the type checker. This means you get no compile time guarantees that ANY of the pointers you’re dealing with aren’t null. Imagine if we had to wrap EVERY Haskell value in a Maybe. We would need to CONSTANTLY unwrap them or else risk tripping a “null-pointer-exception”. In Haskell, once we’ve handled the case once, we can pass on a pure value. This allows other code to know that it will not throw random errors. Consider this example. We check that our pointer in non-null once already in . Despite this, good programming practice dictates that we perform another check in . Nothing function1 function2 public void function1(MyObject obj) {  if (obj == null) {    // Deal with error  } else {    function2(obj);  }}

public void function2(MyObject obj) {  if (obj == null) {    // ^^ We should be able to make this call redundant  } else {    // …  }} The second sticky point comes up when we’re dealing with non-pointer, primitive values. We often don’t have a good way to handle these cases. Suppose your function returns an , but it might fail. How do you represent failure? It’s not uncommon to see side cases like this handled by using a “sentinel” value, like 0 or -1. int But if the range of your function spans all the integers, you’re a little bit stuck there. The code might look cleaner if you use an enumerated type, but this doesn’t avoid the problem. The same problem can even crop up with pointer values if null is valid in the particular context. public int integerSquareRoot(int x) {  if (x < 0) {    return -1;  } else {    return Math.round(Math.sqrt(x));  }}

public void (int a) {  int result = integerSquareRoot(a);  if (result == -1) {    // Deal with error  } else {    // Use correct value  }} Finally, monadic composition with is much more natural in Haskell. There are many examples of this kind of spaghetti in Java code: Maybe public Result computation1(MyObject value) {  …}

public Result computation2(Result res) {  …}

public int intFromResult(Result res) {  …}

public int spaghetti(MyObject value) {  if (value != null) {    result1 = computation1(value);    if (result1 != null) {      result2 = computation2(result1);      if (result2 != null) {        return intFromResult(result2);      }    }  }  return -1;} Now if we’re being naive, we might end up with a not-so-pretty version ourselves: computation1 :: MyObject -> Maybe Resultcomputation2 :: Result -> Maybe ResultintFromResult :: Result -> Int

spaghetti :: Maybe MyObject -> Maybe Intspaghetti value = case value of  Nothing -> Nothing  Just realValue -> case computation1 realValue of    Nothing -> Nothing    Just result1 -> case computation2 result1 of      Nothing -> Nothing      Just result2 -> return $ intFromResult result2 But as we discussed in our first , we can make this much cleaner. We’ll compose our actions within the monad: Monads article Maybe cleanerVersion :: Maybe MyObject -> Maybe IntcleanerVersion value = do  realValue <- value  result1 <- computation1 realValue  result2 <- computation2 result1  return $ intFromResult result2 Using Either Now suppose we want to make our errors contain a bit more information. In the example above, we’ll output if it fails. But code calling that function will have no way of knowing what the error actually was. This might hinder our code's ability to correct the error. We'll also have no way of reporting a specific failure to the user. As we’ve explored, Haskell’s answer to this is the monad. This allows us to attach a value of any type as a possible failure. In this case, we'll change the type of each function. We would then update the functions to use a descriptive error message instead of returning . Nothing Either Nothing computation1 :: MyObject -> Either String Resultcomputation2 :: Result -> Either String ResultintFromResult :: Result -> Int

eitherVersion :: Either String MyObject -> Either String InteitherVersion value = do  realValue <- value  result1 <- computation1 realValue  result2 <- computation2 result1  return $ intFromResult result2 Now suppose we want to try to make this happen in Java. How do we do this? There are a few options I’m aware of. None of them are particularly appetizing. Print an error message when the failure condition pops up. Update a global variable when the failure condition pops up. Create a new data type that could contain either an error value or a success value. Add a parameter to the function whose value is filled in with an error message if failure occurs. The first couple rely on arbitary side effects. As Haskell programmers we aren’t fans of those. The third option would require messing with Java’s template types. These are far more difficult to work with than Haskell’s parameterized types. If we don’t take this approach, we’d need a new type for every different return value. The last method is a bit of an anti-pattern, making up for the fact that tuples aren’t a first class construct in Java. It’s quite counter-intuitive to check one of your values for what do as an an result. So with these options, give me Haskell any day. input output Using Exceptions and Handlers Now that we understand the more “pure” ways of handling error cases in our code, we can deal with exceptions. Exceptions show up in almost every major programming language; Haskell is no different. Haskell has the type that encapsulates possible failure conditions. It can wrap any type that is a member of the typeclass. You'll generally be creating your own exception types. SomeException Exception Generally, we throw exceptions when we want to state that a path of code execution has failed. Instead of returning some value to the calling function, we’ll allow completely different code to handle the error. If this sounds convoluted, that’s because it kind’ve is. In general you want to prefer keeping the control flow as clear as possible. Sometimes though we cannot avoid it. So let’s suppose we’re calling a function we know might throw a particular exception. We can “handle” that exception by attaching a handler. In Java, you do this pattern like so: public int integerSquareRoot(int value) throws NegativeSquareRootException {  ...}

public int mathFunction(int x) {  try {    return 2 * squareRoot(x);  } catch (NegativeSquareRootException e) {    // Deal with invalid result  }} To handle exceptions in this manner in Haskell, you have to have access to the IO monad. The most general way to handle exceptions is to use the function. When you call the action that might throw the exception, you include a “handler” function. This function will take the exception as an argument and deal with the case. If we want to write the above example in Haskell, we should first define our exception type. We only need to derive to also derive an instance for the typeclass: catch Show Exception import Control.Exception (Exception)

data MyException = NegativeSquareRootException  deriving (Show)

instance Exception MyException Now we can write a pure function that will throw this exception in the proper circumstances. import Control.Exception (Exception, throw)

integerSquareRoot :: Int -> IntintegerSquareRoot x  | x < 0 = throw NegativeSquareRootException  | otherwise = undefined While we can throw the exception from pure code, we need to be in the monad to catch it. We’ll do this with the function. We’ll use a handler function that will only catch the specific error we’re expecting. It will print the error as a message and then return a dummy value. IO catch import Control.Exception (Exception, throw, catch)

…mathFunction :: Int -> IO IntmathFunction input = do  catch (return $ integerSquareRoot input) handler  where    handler :: MyException -> IO Int    handler NegativeSquareRootException =       print "Can't call square root on a negative number!" >> return (-1) MonadThrow We can also generalize this process a bit to work in different monads. The typeclass allows us to specify different exceptional behaviors for different monads. For instance, throws exceptions by using . uses , and will use . When we’re in a general function, we throw exceptions with . MonadThrow Maybe Nothing Either Left IO throwIO MonadThrow throwM callWithMaybe :: Maybe IntcallWithMaybe = integerSquareRoot (-5) -- Gives us `Nothing`

callWithEither :: Either SomeException IntcallWithEither = integerSquareRoot (-5) -- Gives us `Left NegativeSquareRootException`

callWithIO :: IO IntcallWithIO = integerSquareRoot (-5) -- Throws an error as normal

integerSquareRoot :: (MonadThrow m) => Int -> m IntintegerSquareRoot x  | x < 0 = throwM NegativeSquareRootException  | otherwise = ... There is some debate about whether the extra layers of abstraction are that helpful. There is a strong case to be made that if you’re going to be using exceptional control flow, you should be using anyway. But using can make your code more extensible. Your function might be usable in more areas of your codebase. I’m not too opinionated on this topic (not yet at least). But there are certainly some strong opinions within the Haskell community. IO MonadThrow Summary Error handling is tricky business. A lot of the common programming patterns around error handling are annoying to write. Luckily, Haskell has several different means of doing this. In Haskell, you can express errors using simple mechanisms like and . Their monadic behavior gives you a high degree of composability. You can also throw and catch exceptions like you can in other languages. But Haskell has some more general ways to do this. This allows you to be agnostic to how functions within your code handle errors. Maybe Either New to Haskell? Amazed by its awesomeness and want to try? Download our ! It has some awesome tools and instructions for getting Haskell on your computer and starting out. Getting Started Checklist Have you tried Haskell but want some more practice? Check out our for some great content and 10 practice problems! Recursion Workbook And stay tuned to the ! Monday Morning Haskell blog

Coping with (Code) Failures

I’ve said it before, but I’ll say it again. As much as we’d like to think it’s the case, our Haskell code doesn’t work just because it compiles. This is why we have . But even if it passes our test suites this doesn’t mean it works as well as it could either. Sometimes we’ll realize that the code we wrote isn’t quite performant enough, so we’ll have to make improvements. test suites But improving our code can sometimes feel like taking shots in the dark. You’ll spend a great deal of time tweaking a certain piece. Then you’ll find you haven’t actually made much of a dent in the total run time of the application. Certain operations generally take longer, like database calls, network operations, and IO. So you can often have a decent idea of where to start. But it always helps to be sure. This is where benchmarking and profiling come in. We’re going to take a specific problem and learn how we can use some Haskell tools to zero in on the problem point. As a note, the tools we’ll use require you to be organizing your code using Stack or Cabal. If you’ve never used either of these before, you should check out our ! It’ll teach you the basics of creating a project with Stack. You’ll also learn the primary commands to use with Stack. It’s brand new and best of all FREE! Check it out! It’s our first course of any kind, so we’re looking for feedback! Stack Mini Course The Problem Our overarching problem for this article will be the “largest rectangle” problem. You can actually try to solve this problem yourself on under the name “John and Fences”. Imagine we have a series of vertical bars with varying heights placed next to each other. We want to find the area of the largest rectangle that we can draw over these bars that doesn’t include any empty space. Here’s a visualization of one such problem and solution: Hackerrank In this example, we have posts with heights . The largest rectangle we can form has an area of 12, as we see with the highlighted squares. [2,5,7,4,1,8] This problem is pretty neat and clean to solve with Haskell, as it lends itself to a recursive solution. First let’s define a couple newtypes to illustrate our concepts for this problem. We’ll use a compiler extension to derive the typeclass on our index type, as this will be useful later. Num {-# LANGUAGE GeneralizedNewtypeDeriving #-}...newtype FenceValues = FenceValues { unFenceValues :: [Int] }newtype FenceIndex = FenceIndex { unFenceIndex :: Int }  deriving (Eq, Num, Ord)-- Left Index is inclusive, right index is non-inclusive newtype FenceInterval = FenceInterval { unFenceInterval :: (FenceIndex, FenceIndex) }newtype FenceSolution = FenceSolution { unFenceSolution :: Int }  deriving (Eq, Show, Ord) Next, we’ll define our primary function. It will take our , a list of integers, and return our solution. FenceValues largestRectangle :: FenceValues -> FenceSolutionlargestRectangle values = ... It in turn will call our recursive helper function. This function will calculate the largest rectangle over a specific interval. We can solve it recursively by using smaller and smaller intervals. We’ll start by calling it on the interval of the whole list. largestRectangle :: FenceValues -> FenceSolutionlargestRectangle values = largestRectangleAtIndices values  (FenceInterval (FenceIndex 0, FenceIndex (length (unFenceValues values))))

largestRectangleAtIndices :: FenceValues -> FenceInterval -> FenceSolutionlargestRectangleAtIndices = ... Now, to break this into recursive cases, we need some more information first. What we need is the index of the minimum height in this interval. One option is that we could make a rectangle spanning the whole interval with this height. i Any other “largest rectangle” won’t use this particular index. So we can then divide our problem into two more cases. In the first, we’ll find the largest rectangle on the interval to the left. In the second, we’ll look to the right. As your might realize, these two cases simply involve making recursive calls! Then we can easily compare their results. The only thing we need to add is a base case. Here are all these cases represented in code: largestRectangleAtIndices :: FenceValues -> FenceInterval -> FenceSolutionlargestRectangleAtIndices  values  interval@(FenceInterval (leftIndex, rightIndex)) =     -- Base Case: Checks if left + 1 >= right    if isBaseInterval interval      then FenceSolution (valueAtIndex values leftIndex)      -- Compare three cases      else max (max middleCase leftCase) rightCase      where      -- Find the minimum height and its index      (minIndex, minValue) = minimumHeightIndexValue values interval      -- Case 1: Use the minimum index      middleCase = FenceSolution $ (intervalSize interval) * minValue      -- Recursive call #1      leftCase = largestRectangleAtIndices values (FenceInterval (leftIndex, minIndex))      -- Guard against case where there is no "right" interval      rightCase = if minIndex + 1 == rightIndex        then FenceSolution (maxBound :: Int) -- Supply a "fake" solution that we'll ignore        -- Recursive call #2        else largestRectangleAtIndices values (FenceInterval (minIndex + 1, rightIndex)) And just like that, we’re actually almost finished. The only sticking point here is a few helper functions. Three of these are simple: valueAtIndex :: FenceValues -> FenceIndex -> IntvalueAtIndex values index = (unFenceValues values) !! (unFenceIndex index)

isBaseInterval :: FenceInterval -> BoolisBaseInterval (FenceInterval (FenceIndex left, FenceIndex right)) = left + 1 >= right

intervalSize :: FenceInterval -> IntintervalSize (FenceInterval (FenceIndex left, FenceIndex right)) = right - left Now we have to determine the minimum on this interval. Let’s do this in the most naive way, by scanning the whole interval with a fold. minimumHeightIndexValue :: FenceValues -> FenceInterval -> (FenceIndex, Int)minimumHeightIndexValue values (FenceInterval (FenceIndex left, FenceIndex right)) =  foldl minTuple (FenceIndex (-1), maxBound :: Int) valsInInterval  where    valsInInterval :: [(FenceIndex, Int)]    valsInInterval = drop left (take right (zip (FenceIndex <$> [0..]) (unFenceValues values)))    minTuple :: (FenceIndex, Int) -> (FenceIndex, Int) -> (FenceIndex, Int)    minTuple old@(_, heightOld) new@(_, heightNew) =      if heightNew < heightOld then new else old And now we’re done! Benchmarking our Code Now, this is a neat little algorithmic solution, but we want to know if our code is efficient. We need to know if it will scale to larger input values. We can find the answer to this question by writing benchmarks. Benchmarks are a feature we can use in conjunction with Cabal and Stack. They work a lot like test suites. But instead of proving the correctness of our code, they’ll show us how fast our code runs under various circumstances. We’ll use the library to do this. We’ll start by adding a section in our .cabal file for this benchmark: Criterion benchmark fences-benchmarks  type:                exitcode-stdio-1.0  hs-source-dirs:      benchmark  main-is:             fences-benchmark.hs  build-depends:       base                     , FencesExample                     , criterion                     , random  default-language:    Haskell2010 Now we’ll make our file , make it a module and add a function. We’ll generate 6 lists, increasing in size by a factor of 10 each time. Then we’ll create a benchmark group and call the function on each situation. fences-benchmark.hs Main main bench module Main where

import Criterionimport Criterion.Main (defaultMain)import System.Random

import Lib

main :: IO ()main = do  [l1, l2, l3, l4, l5, l6] <- mapM     randomList [1, 10, 100, 1000, 10000, 100000]  defaultMain    [ bgroup "fences tests"       [ bench "Size 1 Test" $ whnf largestRectangle l1      , bench "Size 10 Test" $ whnf largestRectangle l2      , bench "Size 100 Test" $ whnf largestRectangle l3      , bench "Size 1000 Test" $ whnf largestRectangle l4      , bench "Size 10000 Test" $ whnf largestRectangle l5      , bench "Size 100000 Test" $ whnf largestRectangle l6      ]    ]

-- Generate a list of a particular sizerandomList :: Int -> IO FenceValuesrandomList n = FenceValues <$> (sequence $ replicate n (randomRIO (1, 10000 :: Int))) We’d normally run these benchmarks with (or if you’re not using Stack). But we can also compile our code with the flag. This will automatically create a profiling report with more information about our code. Note using profiling requires re-compiling ALL the dependencies to use profiling as well. So you don't want to switch back and forth a lot. stack bench cabal bench --profile >> stack bench --profileBenchmark fences-benchmarks: RUNNING...benchmarking fences tests/Size 1 Testtime                 47.79 ns   (47.48 ns .. 48.10 ns)                     1.000 R²   (0.999 R² .. 1.000 R²)mean                 47.78 ns   (47.48 ns .. 48.24 ns)std dev              1.163 ns   (817.2 ps .. 1.841 ns)variance introduced by outliers: 37% (moderately inflated)

benchmarking fences tests/Size 10 Testtime                 3.324 μs   (3.297 μs .. 3.356 μs)                     0.999 R²   (0.999 R² .. 1.000 R²)mean                 3.340 μs   (3.312 μs .. 3.368 μs)std dev              98.52 ns   (79.65 ns .. 127.2 ns)variance introduced by outliers: 38% (moderately inflated)

benchmarking fences tests/Size 100 Testtime                 107.3 μs   (106.3 μs .. 108.2 μs)                     0.999 R²   (0.999 R² .. 0.999 R²)mean                 107.2 μs   (106.3 μs .. 108.4 μs)std dev              3.379 μs   (2.692 μs .. 4.667 μs)variance introduced by outliers: 30% (moderately inflated)

benchmarking fences tests/Size 1000 Testtime                 8.724 ms   (8.596 ms .. 8.865 ms)                     0.998 R²   (0.997 R² .. 0.999 R²)mean                 8.638 ms   (8.560 ms .. 8.723 ms)std dev              228.8 μs   (193.6 μs .. 272.8 μs)

benchmarking fences tests/Size 10000 Testtime                 909.2 ms   (899.3 ms .. 914.1 ms)                     1.000 R²   (1.000 R² .. 1.000 R²)mean                 915.1 ms   (914.6 ms .. 915.8 ms)std dev              620.1 μs   (136.0 as .. 664.8 μs)variance introduced by outliers: 19% (moderately inflated)

benchmarking fences tests/Size 100000 Testtime                 103.9 s    (91.11 s .. 117.3 s)                     0.997 R²   (0.997 R² .. 1.000 R²)mean                 107.3 s    (103.7 s .. 109.4 s)std dev              3.258 s    (0.0 s .. 3.702 s)variance introduced by outliers: 19% (moderately inflated)

Benchmark fences-benchmarks: FINISH So when we run this, we’ll find something…troubling. It takes a looong time to run the final benchmark on size 100000. On average, this case takes over 100 seconds…more than a minute and a half! We can further take note of how the average run time increases based on the size of the case. Let’s pare down the data a little bit: Size 1: 47.78 nsSize 10: 3.340 μs (increased ~70x)Size 100: 107.2 μs (increased ~32x)Size 1000: 8.638 ms (increased ~81x)Size 10000: 915.1 ms (increased ~106x)Size 100000: 107.3 s (increased ~117x) Each time we increase the size of the problem by a factor of 10, the time spent increased by a factor closer to 100! This suggests our run time is (check out if you are unfamiliar with Big-O notation). We’d like to do better. O(n^2) this guide Determining the Problem So we want to figure out why our code isn’t performing very well. Luckily, we already profiled our benchmark!. This outputs a specific file that we can look at, called . It has some very interesting results: fences-benchmark.prof COST CENTRE                            %time %allocminimumHeightIndexValue.valsInInterva  69.8   99.7valueAtIndex                           29.3    0.0 We see that we have two big culprits taking a lot of time. First, there is our function that determines the minimum between a specific interval. The report is even more specific, calling out the specific offending part of the function. We spend a lot of time getting the different values for a specific interval. In second place, we have . This means we also spend a lot of time getting values out of our list. valueAtIndex First let’s be glad we’ve factored our code well. If we had written our entire solution in one big function, we wouldn’t have any leads here. This makes it much easier for us to analyze the problem. When examining the code, we see why both of these functions could produce behavior. O(n^2) Due to the number of recursive calls we make, we’ll call each of these functions times. Then when we call , we use the operator on our linked list. This takes time. Scanning the whole interval for the minimum height has the same effect. In the worst case, we have to look at every element in the list! I’m hand waving a bit here, but that is the basic result. When we call these pieces times, we get time total. O(n) valueAtIndex (!!) O(n) O(n) O(n) O(n^2) Cliff Hanger Ending We can actually solve this problem in time, a dramatic improvement over the current . But we’ll have to improve our data structures to accomplish this. First, we’ll store our values so that we can go from the index to the element in sub-linear time. This is easy. Second, we have to determine the index containing the minimum element within an arbitrary interval. This is a bit trickier to do in sub-linear time. We'll need a more advanced data structure. O(n log n) O(n^2) To find out how we solve these problems, you’ll have to wait for part 2 of this series! Come back next week to the ! Monday Morning Haskell blog As a reminder, we’ve just published a FREE . It’ll teach you the basics of laying out a project and running commands on it using the Stack tool. You should enroll in the to sign up! This is our first course of any kind, so we would love to get some feedback! Once you know about Stack, it’ll be a lot easier to try this problem out for yourself! mini-course on Stack Monday Morning Haskell Academy In addition to Stack, recursion also featured pretty heavily in our solution here. If you’ve done any amount of functional programming you’ve seen recursion in action. But if you want to solidify your knowledge, you should download our FREE ! It has two chapters worth of content on recursion and it has 10 practice problems you can work through! Recursion Workbook Never programmed in Haskell? No problem! You can download our and it’ll help you get Haskell installed. It’ll also point you in the direction of some great learning resources as well. Take a look! Getting Started Checklist is how hackers start their afternoons. We’re a part of the family. We are now and happy to opportunities. Hacker Noon @AMI accepting submissions discuss advertising & sponsorship To learn more, , , or simply, read our about page like/message us on Facebook tweet/DM @HackerNoon. If you enjoyed this story, we recommend reading our and . Until next time, don’t take the realities of the world for granted! latest tech stories trending tech stories

How well does it work? Profiling in Haskell

I am very excited, not only because it is my birthday today, but also because I am announcing the first release of HiddenWallet, which I believe is a major step down the road of making Bitcoin anonymous again. Download and read more about HiddenWallet on GitHub! What is TumbleBit and HiddenWallet? In a nutshell is a trustless Bitcoin mixer, where the tumbler cannot steal your coins, nor it can deanonymize you. Why does TumbleBit need a dedicated wallet? Why not just integrate it to Electrum or to Blockchain.info or to some other wallet? The problem is, Why does a full node provide more privacy, than a normal Bitcoin wallet? At first glance, they are leaving similar tracks on The Blockchain, but that’s not the only aspect of your privacy you want to protect. Blockchain surveillance companies or various third parties can reestablish the links between your wallet addresses by combining blockchain and network analysis techniques. This is not just theoretical threat, it is reality today and they are in this. If you are using a full node as your personal Bitcoin wallet, you defeat the possibility of network analysis, because it doesn’t only asks for data from third parties that it needs, like address balances, but asks for all the a data the Bitcoin network have, like all the blocks, so the other nodes cannot figure out which data it needs. You can on this topic, if you want to dive deeper. TumbleBit I would like to also point out TumbleBit is not integrated yet, it’s the next step on the roadmap. if the wallet, what your peers are using in the mix is screwed up from a privacy point of view, even if you are using TumbleBit in the correct way with a fullnode, your anonymity set is screwed up, because of your peers. extremely successful read my research However running a Bitcoin full node is cumbersome and it is expected to get much worse over time as Bitcoin is pressured to scale, but we can do better, if we our only requirement is to keep the full node level privacy, that’s where HiddenWallet and full block downloading SVP wallets come in.Interestingly this is the type of SPV wallet that Satoshi described in his whitepaper, but none of today’s SPV wallets implemented. HiddenWallet doesn’t store the blockchain and doesn’t validate the blocks, rather it’s using merkle proofs, like other SPV wallets to make sure a transaction is valid. Unlike other SPV wallets and like full nodes it downloads full blocks, so an curious listener has no idea what information we are interested in, therefore it cannot reestabilish the link between our wallet addresses. This is the kind of privacy that is needed for the ideal working of TumbleBit. Roadmap — This feature provides full node level privacy against network analysis with SPV security. Full Block Downloading SPV Ready, unstable. — TumbleBit is a Bitcoin mixer where not even the Tumbler can steal your coins, nor deanonymize you. TumbleBit integration, Classic Tumbler Mode Next up. — 10–100x performance increase, while still keeping full node level privacy against network analysis, but replacing SPV security to trusted full node security. Transaction Filtered Block Downloading Release stable version. TumbleBit Paymen Hub Mode? JoinMarket integration? Stealth addresses support?

Introducing HiddenWallet : Full Block SPV TumbleBit Wallet — Testing Release

Seamless Moves: RGB++ Ports Assets Across Blockchains Without Cross-Chain Bridges

বিরামহীন চালনা: ক্রস-চেইন ব্রিজ ছাড়াই ব্লকচেইন জুড়ে RGB++ পোর্টের সম্পদ

Nahtlose Übertragung: RGB++ portiert Assets über Blockchains ohne Cross-Chain-Brücken

Movimientos fluidos: RGB++ transfiere activos a través de cadenas de bloques sin puentes entre cadenas

Déplacements fluides : les actifs des ports RGB++ sur les blockchains sans ponts entre chaînes

निर्बाध चालें: RGB++ क्रॉस-चेन ब्रिज के बिना ब्लॉकचेन में परिसंपत्तियों को पोर्ट करता है

シームレスな移動: RGB++ はクロスチェーン ブリッジなしでブロックチェーン間で資産を移植します

원활한 이동: 교차 체인 브리지 없이 블록체인 전반에 걸쳐 RGB++ 포트 자산

Movimentos contínuos: RGB++ transporta ativos em blockchains sem pontes entre cadeias

Бесшовные перемещения: RGB++ портирует активы между блокчейнами без межцепочных мостов

Kesintisiz Hareketler: RGB++ Bağlantı Noktaları Varlıkları Zincirler Arası Köprüler Olmadan Blok Zincirleri Arasında Taşır

Di chuyển liền mạch: Cổng RGB++ tài sản trên các chuỗi khối mà không cần cầu nối chuỗi chéo

无缝迁移：RGB++ 无需跨链桥即可跨区块链移植资产

Stories, News & Mentions about INZINT

Trending Topics

Classic

Neon Noir

Minty

Newspaper

HN StartUps