Soroco

soroco

This article also comes in video form! We’ll assume that you’ve completed the steps from the first part of our eScooter series already. For a broader overview of this project, check out the first article here . So you’ve set up your IoT eScooter with in , sent scooter data to the cloud with in , and remotely managed your scooter with in . So far, we’ve only been collecting data and managing our devices. In this fourth and final article of the series, we’ll to our IoT eScooter with SORACOM Air part one SORACOM Harvest part two SORACOM Napter part three link everything together with an Android app and add surge pricing capabilities SORACOM Funk. With a basic surge pricing model, we can somewhat accurately reflect the current scooter supply and demand in the area and adapt our scooters' prices accordingly. With our model, a scooter’s price per minute will rise if more scooters are already in use around it. Although we created a way to send data from our Raspberry Pi in the last article, we still need a way to calculate prices in realtime. Normally this would require a lot of processing power on the Raspberry Pi, but we can offload this time and power-intensive logic onto Lambda using SORACOM Funk. It essentially streamlines the process of passing data from the eScooter-mounted Raspberry Pi and a serverless compute instance like AWS Lambda. Instead of spending our time on setting up more functionality on Lambda, we’ll focus on setting up Funk, which is much easier. We’ll be using Amazon Web Services’ (AWS) serverless platform, . We can use it to create Lambda functions, which scale automatically as our userbase grows. What this means for you is that you only pay for the compute time you actually use, so you don’t need to think about server usage beforehand. Basically, Lambda lets you focus on writing code rather than managing servers and costs. AWS Lambda To get started with Lambda, make an AWS account if you don’t already have one. We’ll be using Node.js with the Serverless platform for our Lambda functions, but do note that you can also run other languages with Lambda. Since we’re using Node.js in this article, make sure that you have it installed on your computer beforehand. If you don’t, follow for Node.js. You can then verify your installation with this link $ node -v which will print out your currently installed version of Node.js. Create a DocumentDB Cluster We’ll first set up a database instance to store our scooters’ data so that our app can access it later. We’re going to use : a NoSQL database to store scooter data and payment transaction information. MongoDB To connect to MongoDB, we’ll use AWS’s dedicated NoSQL service: . You can access it from your AWS console . DocumentDB here Under DocumentDB on the console, click . Now give your DocumentDB cluster a unique identifier. Change the number of instances to what you feel fit. We set it to 1 because fewer instances means less cost. For authentication, come up with a username and password. This will be used to connect to the DocumentDB cluster manually and programmatically. Finally, click . Create Create Cluster After AWS finishes creating your cluster, you will be presented with a page of commands like $ mongo --ssl --host scooter-db.[redacted] : --sslCAFile rds-combined-ca-bundle --username scooteradmin --password <insertYourPassword> .us-east-1 .docdb .amazonaws .com 27017 .pem and mongod //scooteradmin: @scooter-db.[redacted].us-east- .docdb.amazonaws. : /?ssl=true&ssl_ca_certs=rds-combined- -bundle.pem&replicaSet=rs0 b: <insertYourPassword> 1 com 27017 ca In this page, under Security Groups, click . In the tabs below, click and to match this Default Inbound Edit This will ensure that your Lambda functions will be able to connect to DocumentDB. Optionally, you can verify your DocumentDB setup by creating an AWS EC2 instance under the same region as the DocumentDB cluster with the MongoDB client installed and run the command from earlier. Get it with this command: mongo You’ll need to download DocumentDB’s public key for the command to work. $ wget http //s3.amazonaws. /rds-downloads/rds-combined- -bundle.pem s: com ca After you’ve completed this step, your MongoDB database should be set up for storing your scooters’ records. Create Lambda Functions Now that we have a database on AWS, we can begin working on the Lambda function. It will be responsible for the functionality of what we have on AWS, including creating/updating database entries, interacting with SORACOM Funk, and serving data to our app. We’ll begin by installing , which is a Node.js tool that helps us deploy serverless functions quickly. To install , run serverless serverless $ npm install -g serverless To verify your installation, run $ serverless If you get an error, you may have to restart your terminal window or rerun the installation command. Now link the tool to AWS. Go to IAM (Identity & Access Management) from your AWS Dashboard. Click and . Name the user something like “serverless-admin” and enable programmatic access. Users Add User Hit next and . Check and click Create User. Attach existing policies directly AdministratorAccess Now, it should show your new user’s API credentials. With these credentials, enter this in your terminal with your Access ID replacing and API Secret replacing {access_id} {secret_key} $ serverless credentials --provider aws --key {access_id} --secret {secret_key} config Great! Your serverless-cli is all setup! Download the serverless project with $ git ://github.com/DevinMui/soracom-escooter-server clone https Switch into the project directory $ soracom-escooter-server cd Install the dependencies $ npm install Install the DocumentDB public key $ wget http //s3.amazonaws. /rds-downloads/rds-combined- -bundle.pem s: com ca Connecting to Stripe to Securely Handle Payments We’ll use Stripe to collect payments from our customers. Stripe gives us a simple-to-use yet secure payment API, which we can use in our server for a small fee. First, create a Stripe account if you haven’t done so already. In your Stripe dashboard, under , click . Copy your . You can also use your test key by flipping the switch to view test data, but doing so will prevent Stripe from charging your users’ credit cards until you disable test mode. Developers API keys Secret key This secret key essentially acts as a password to calling Stripe’s API through code. Make sure to keep it in a safe place and never share it, because anyone who has access to this key could get access to your payments through Stripe. If this happens, you should request a new key through the same API keys page. Once you have your secret API key, you can start using Stripe! Keep the key on hand, because we’ll need to add it to our Lambda function in the next step. Configure serverless.yml Although we already have the serverless code for the Lambda function written for you already, we’ll need to create an additional file before we deploy our code to AWS. This file will store vital settings and keys (i.e. the Stripe API key). Because of how important this file is, you should keep it hidden from the public (which is why it is not included in the GitHub repository). serverless.yml Create a new file . This file will hold all our environmental variables, which can be accessed through the Lambda function’s code. Add the following lines into the file, replacing with your Stripe API key and with the MongoDB URL from the steps above. Be sure to follow the format of the code below exactly or AWS might reject the file env.yml {stripe_secret_key} {mongo_url} default: STRIPE_SECRET_KEY: {stripe_secret_key} MONGO_URL: {mongo_url}/test?ssl=true&replicaSet=rs0% Now that we have that file configured, deploy the application with $ serverless deploy Now if you try to access the function, you may notice the request will timeout. This is because Lambda is not set under the same Virtual Private Cloud (VPC) as DocumentDB and therefore has no access to DocumentDB. getScooters Go to your AWS Dashboard. Click . Under the section, click . Services Compute Lambda Click on one of your scooter Lambda functions. Scroll down until you see the panel. Under the VPC selector, click the one associated with your DocumentDB cluster. Under and , copy two of each ID (e.g. subnet-37fca853 and sg-163e825e). Network Subnets Security Groups Now update your file to update the VPC settings to use the correct subnet IDs and security group IDs. Under line 10, replace the security group IDs listed with your security group IDs. Under line 13, replace the subnet IDs with your subnet IDs. serverless.yml vpc: securityGroupIds: - # your first security group id - # your second security group id subnetIds: - # your first subnet id - # your second subnet id - # your third subnet id iamRoleStatements: Redeploy your Lambda functions. $ serverless deploy Open the link associated to the function. If the server responds with a , it worked! Your functions are now connected to DocumentDB. getScooters [] Set up NAT Unfortunately, by setting up DocumentDB, Lambda won’t be able to connect to the outside Internet anymore. However, the solution is trivial to implement. We just need to change our NAT and routing settings. Create a new pointing to the public subnet. NAT Gateway Create a pointing to the Routing Table 0.0.0.0/0 Internet Gateway Find the public subnet and under , click Actions Edit route table association Create two new subnets. It may be useful to label these two as private subnets. Create another . This time point to the we created earlier. Routing Table 0.0.0.0/0 NAT Gateway On both private subnets, edit the route associations to use this new routing table. For more information on how this works or for troubleshooting, take a look at . AWS’s official documentation/support Funk Up Your Pricing Model To make our operation more profitable, we can add surge pricing to our fleet to reflect the current scooter supply and demand in the area. This way, a scooter’s price per minute will rise if there are other scooters in use around its area. We can easily achieve this using SORACOM Funk which streamlines the connection between our cellular IoT device and AWS Lambda. We’ll have to create another in AWS to grant Soracom access to our function first. Like before, open the IAM dashboard and add a new user. Enter a name for the user like and check IAM user soracom-funk Programmatic access. In the next screen, click and click the button. Under the JSON tab, copy paste the following. Attach existing policies directly Create policy { : , : [
    { : , : , : , : }
  ]
} "Version" "2012-10-17" "Statement" "Sid" "VisualEditor0" "Effect" "Allow" "Action" "lambda:InvokeFunction" "Resource" "<Lambda function ARN of the createOrUpdateScooter function>" Make a name for the policy and click the button. Go back to the screen, find the policy you just created, and click its checkbox. Click the buttons until the user is created. It is imperative that you . If you do, you will have to restart recreating the IAM user process as the secret key is only viewable immediately after the creation of a new IAM user. Create policy Add user Next do not close this window As with all Soracom products, click on the group you want to enable SORACOM Funk. Find the SORACOM Funk panel and turn Funk on. Change the service the , content type to , and the function ARN to the Lambda function ARN of the function. AWS Lambda JSON createOrUpdateScooter For the credentials, you’ll have to click . Change the to a label you can identify. From the IAM user information page, paste in the and and click Add new credentials Credentials set ID AWS Access key ID Secret access key Register With those few steps, SORACOM Funk has now been set up. When you make an HTTP request to , this request will now be forwarded instead to your Lambda function. This forwarding is pretty powerful and allows developers to offload processor/battery intensive operations from IoT devices to AWS Lambda instead. For this article, we’ll offload the algorithm to calculate the surge price of the scooter from the Raspberry Pi to the cloud. If you take a look at the function, you can see that the algorithm may take a while to run depending on how many scooters are active and how many scooters are within the current scooter’s location, making this a realistic example of how SORACOM Funk can reduce computational load on cellular devices. http://funk.soracom.io createOrUpdateScooter Next, we’ll want to set up our cellular device to connect to SORACOM Funk. SSH into your Raspberry Pi. Clone the repository if you have not been following along with our previous articles. $ git ://github.com/DevinMui/soracom-escooter-device clone https Switch into the project directory. $ soracom-escooter-device cd Install the necessary Python modules. $ pip install -r requirements.txt On line 7 of , update the variable to reflect your scooter’s Bluetooth MAC address. Finally, start the program. funk.py mac $ python server.py Your scooter will now send data over to the SORACOM Funk endpoints every five seconds! To test this, you can create a front end solution to call the other Lambda functions to lock/unlock the scooter you can use our React Native app below to test it out. You should see the location of the scooter visualized on the map. OR Run the App If you don’t want to spend more time creating a new app for your user, you can simply use our example app. It currently runs only on Android and relies on React Native. As a result, you’ll need to have Android Studio setup for React Native (which you can do by following ). There’s also a small setup process for the app that you’ll need to complete if you’re building it for your own machine. the official React Native documentation here First clone the repository $ git ://github.com/aaronkh/soracom-escooter-app.git clone https and install the required modules with $ npm install at the root of the cloned repository. Since this project relies on Google Maps, we’ll need a Google Maps API key. You should follow the official documentation for doing so . Like the Stripe key, keep this key hidden from the public. You can do so by hiding it in your file. This file is found under on both Mac and Windows. If it doesn’t already exist create one named . Add into the file, replacing with your own API key. You can optionally restrict access to only allow your app in your Google Cloud console, but we’ll skip that for now. here gradle.properties /Users/{Username}/.gradle gradle.properties GMAPS_SDK_KEY="{YOUR_API_KEY}" {YOUR_API_KEY} Next we’ll set up a connection to Lambda through the app. To do so, simply add the base URL to your Lambda function as an environment variable named . Do so with API_URL $ API_URL={YOUR_URL} export on Mac/Linux or > setx API_URL "{YOUR_URL}" on Windows, again replacing with your Lambda URL. {YOUR_URL} If you forgot the URL or lost it, simply run and it should print the invocation URL to your console. $ serverless deploy Lastly, we’ll need to add our Stripe API key to the app. Open up the on Stripe and copy the publishable key under the Standard Keys table. Make sure that you are using and . This publishable key will be visible to users who download the app, but it’s useless without your secret key, which is hidden to your users in Lambda’s file. Create an environment variable named with API keys page the publishable key not the secret key env.yml STRIPE_KEY $ STRIPE_KEY={YOUR_KEY} export on Mac/Linux or > setx STRIPE_KEY "{YOUR_KEY}" on Windows, again replacing with your Stripe key. {YOUR_KEY} Finally, we can install the app onto an Android device by running $ react-native run -android at the root of the repository. Conclusion After the long setup process for AWS Lambda, you should now be able to access both the state and pricing of every electric scooter in your fleet. Combined with the steps in our previous articles, you should also be able to lock and unlock your scooter from anywhere in the world and see if any scooters need recharging. We even have a user-facing Android app to help users locate electric scooters around them and pay for their rides. Plus, with our surge pricing model and AWS’s auto-scaling Lambda functions on the cloud, your electric scooters’ prices will adapt to your platform’s demand, guaranteeing increased revenues as your user base grows. Now we have all the tools needed to kickstart your new IoT eScooter startup! In this four-part series, we’ve explored how to build a cellular connected IoT eScooter using Soracom’s services in depth. We learned how to connect devices to the cloud through a cellular connection with , how to collect and visualize scooter data with and , how to remotely connect and troubleshoot your scooter with , and how to offload computationally complex logic from our device to the cloud through . SORACOM Air SORACOM Harvest Lagoon SORACOM Napter SORACOM Funk All in all, creating an IoT eScooter was much easier than we’d imagined. We expected to spend more hours reading documentation and writing code to connect our electric scooter to the cloud. However, we found that the development time was greatly reduced because of the Soracom platform’s simplicity. Most of the Soracom integrations took only minutes to set up and only required us to add a single HTTP request into our code. We’ve only scratched the surface of what the SORACOM platform is capable of. They offer dozens of other easy-to-use products and services, and I’m sure you makers can come up with new and unique ways to leverage the SORACOM platform to build your next game-changing IoT project.

How We Added Surge Pricing Capabilities to our IoT eScooter with AWS Lambda and SORACOM Funk

We’ll assume that you’ve completed the steps from the first part of our eScooter series already. For a broader overview of this project, check out the first article . here In the , we were able to get our IoT eScooter sending data to the SORACOM cloud within just a few minutes. After we collect our data, we will need a way to remotely control and manage our eScooter based on the information we collected. previous article To do that, we need a tool to access and manage our IoT devices. It also needs to be secure despite a remote connection, so that it will be difficult for malicious hackers to tamper with the electric scooter but also allow us to send commands to it regardless of location. Once again, Soracom’s diverse range of services has one suited for our purpose: . This service lets us expose a TCP port for a limited time through the SORACOM Air SIM card, which we’ll use to control the connected Raspberry Pi. SORACOM Napter In this article, we’ll walk you through how we set up Napter on the Raspberry Pi, and how we used it to remotely manage the electric scooter. SSH Setup with SORACOM Napter To manage a Raspberry Pi on our local network, we could issue commands to it via SSH without much trouble. However, our fleet of cellular-connected electric scooters will need to be connected to a number of different cellular towers rather than on our local WiFi network. Napter enables us to open ports to communicate with our cellular connected devices. We can use a TCP protocol like SSH or HTTP to communicate with the Raspberry Pi from our computer. This allows us to set up remote maintenance, troubleshooting, and other remote access tasks on our IoT eScooter without traditional relay servers or software agents. For our project, we’ll use Napter to connect to our scooter via SSH. We’ll also expose a port for HTTP purposes for remote maintenance and troubleshooting purposes through a more visual interface than the terminal. Like the other Soracom services we’ve used so far, Napter’s setup is quite simple. Rather than spending hours deploying relay servers, we only need a few clicks to get communications with our device up and running. We don’t even need to write any code! On the Soracom console, simply click the SIM you would like remote access to. Click the button to reveal a drop down menu. Actions Click On-demand Remote Access. This will open a dialog where you can request a port to be forwarded. Here, we will choose to forward port 22, as it is the standard SSH port. Under , enter . You can also change the timeout so that the port will automatically close after a certain length of time. This will make sure someone else using your computer after you cannot access the Raspberry Pi through SSH after the port times out. To further limit access to the port, you can enter in an IP address range. Port to expose 22 Access allowed for Since an IP address identifies your computer to other computers, entering an address or address range here will restrict access to only those computers whose IP address fall within the range. This field defaults to the IP address of the computer used to connect to the website, so we’ll leave it blank. You can add more addresses in this field if you want another user to have access to the Raspberry Pi’s SSH (i.e. another admin). After that’s finished, the dialog will display a SSH command. Now, let’s connect to our Raspberry Pi through SSH with this link. Copy and pasting the above command into our computer’s terminal will let us SSH into the Raspberry Pi if it is connected to Soracom’s cellular network. Note that you’ll have to change user to pi to login as the pi user on the Raspberry Pi. $ ssh -p {port} pi@ {ip} Now we can start up our Python script and manage the filespace from anywhere in the world without needing to be on the same WiFi network! Creating a Web Interface for the Raspberry Pi What if we just want a simple way to manage and view scooter data without the terminal? For example, we might want to make a mobile app for our users later on, and most phones do not have access to a shell to type commands into. Luckily, SORACOM Napter can also expose ports for HTTP in addition to SSH. Since we’ve finished setting up SORACOM Napter in the previous step, we can now work on adding some functionality to the Raspberry Pi. The code we’ll use below isn’t part of Soracom’s setup, but it’s what we’re going to use to communicate with Napter. It uses HTTP, which is a standardized protocol for sending data over the internet. If you already have an existing app which utilizes HTTP requests and a server on the Raspberry Pi, feel free to skip this step. As long as its connected to the SORACOM Air SIM, it should already be accessible! HTTP lets us send and receive requests from a server hosted on the Raspberry Pi. With HTTP, we can send HTML to visualize scooter information rather than reading through SSH logs. Though we could easily build an app for the electric scooter in this step, we’ll save that for the next article and instead focus on setting up the server below. If you’ve been following along and already cloned the GitHub repository, feel free to skip this step. Otherwise open up a terminal or SSH into the Raspberry Pi and enter $ git @github.com:DevinMui/soracom-escooter-device.git clone git and install the requirements $ pip install -r requirements.txt In this repository, we have a set of scripts that connect to different Soracom services. For the web server, we can use the aptly named script to run a Flask web server on port 5000. napter.py $ python3 napter.py With this command, we’ll now have a web server up and running! Using it, we’ll be able to control our scooter by sending HTTP requests and view scooter data. will unlock our scooter, will lock our scooter, and will show the scooter information. Neat! GET /unlock GET /lock GET / Lock/Unlock the Scooter with Napter Let’s set up another remote access configuration with SORACOM Napter. This time, we’ll expose the Flask server so we can access it from our computer Now just visit the address given by Napter on screen by copy-pasting it into your web browser http: //{ip} You should see a simple web page with a list of information (GPS coordinates, battery, and speed). On the scooter, there are options to lock the device which prevents the scooter from moving. This would be useful to avoid scooter thefts and to prevent non customers from accessing the device and draining its battery. Luckily, the Bluetooth interface exposes this lock feature which we take advantage of in the program. To lock your device, you can send an HTTP request to the . For example, using cURL GET http://{ip}/lock curl / $ http: /{ip}/lock To unlock your device, you can send the same request to . http://{ip}/unlock curl / $ http: /{ip}/unlock Note that an HTTP GET request is the same as visiting the web page, so if you’re unfamiliar with HTTP requests, you can simply open the page in a browser. Conclusion Using a browser, you should now be able to lock/unlock your eScooter and view its current battery, location, and speed. We used SORACOM Napter to expose an HTTP port so we can remotely access a web server on the Raspberry Pi. We also exposed the Raspberry Pi’s SSH port securely, so we could start/stop scripts on our eScooter from anywhere in the world. Because of its ease of use, SORACOM Napter has proven to be a useful service for remotely managing our cellular-connected IoT eScooter. We were able to access our server in just a few minutes. Most importantly, . Security is always a serious and difficult challenge when building an IoT device, and the electric scooter is no exception. Napter takes care of security for us Napter’s ability to give only a select few IP ranges access to SSH and HTTP gives us ease of mind; we don’t have to worry about hackers breaking into the electric scooter and running commands without our consent. We could see a service like Napter being used in other IoT applications where security is paramount, like in asset monitoring and payment systems. In the next and last article of our four part series, we’ll demonstrate how all these services can work in conjunction with one another. We’ll create an AWS Lambda application hooked up to a MongoDB database to communicate with our electric scooter and wrap the series up with a client-side app built with React Native to enable surge pricing on scooters. See you next time!

How to Control your IoT Electric Scooter Fleet Remotely with SORACOM Napter [Part 3]

With the of ride sharing e-scooter companies like Lime and Bird, we were curious how hard it would be to build a working prototype using the same scooter they do (Xiaomi Mijia M365 Electric Scooter). growth and visibility We ( and I) recently participated in the sponsored by Soracom where we learned about Soracom’s cellular connectivity, management console, and several advanced features (cloud integration, data visualization, and remote management).  We realized that using Soracom, a Raspberry Pi, and GPS sensor, we could create a comprehensive working prototype of an eScooter company very easily and very quickly. Devin Hackster.io Cellular IoT Challenge In this multi-part series, we’ll be documenting how we built key components of a cellular IoT electric scooter company, roughly following the outline below. We’ll walk you through each step of our development process so that you can build your own cloud-connected electric scooter Connecting an electric scooter to the cloud with a cellular modem, SIM card, and Raspberry Pi Sending electric scooter data to the cloud and visualizing it Remotely managing the electric scooter Optimizing electric scooter operations and profits with serverless cloud functions Here’s what you’ll need to follow along: 1x - $399.00 Xiaomi Mijia M365 Electric Scooter 1x - $109.00 SORACOM Cellular IoT Starter Kit 1x - $15.98 GPS module We chose the Xiaomi electric scooter because it is both (Bird, Lime, etc.) and because it can be easily controlled via Bluetooth. However, to access these controls long-distance, we needed some kind of connection to the internet. We found that the only real solution for an eScooter company was to use a cellular connection, which would ensure access from anywhere in the world. So, the Soracom kit was a perfect match. Included is a SIM card with global coverage and a Raspberry Pi, which has a built-in Bluetooth antenna and is easy to set up. widely used by existing eScooter companies The Raspberry Pi acts as the “brains” of the eScooter. Not only will it communicate directly with the electric scooter’s Bluetooth controls, but it will also report various data points (location, battery, usage) to the Soracom cloud. Using the provided Soracom services for sending and displaying data, we can easily manage the electric scooter remotely. The data is sent securely through SORACOM Harvest (one of Soracom’s many products available for their SIMs) and stored in the Soracom cloud. This means that it is very difficult for eScooter data to be erased, allowing us to better keep track of exactly where the electric scooter’s been. The GPS module here makes up for the lack of location tracking built into the electric scooter. Though the SIM card automatically gathers location data through its cellular network, its location estimate is very inaccurate. We attached an inexpensive GPS module onto the Raspberry Pi for more accurate tracking. Raspberry Pi Setup Setting up the Raspberry Pi in the next two steps will turn our normal Xiaomi electric scooter into a smart eScooter by connecting it to the cloud. Communications will pass through the Raspberry Pi, and it will be responsible for commanding the electric scooter through Bluetooth. The SORACOM Cellular IoT Starter Kit includes a Raspberry Pi Zero W, which fits the job perfectly. It is essentially a low-power computer with many different wireless connection options and a fairly easy setup process. The Raspberry Pi that came with our kit didn’t work straight out of the box, so there’s still a couple steps to do before that can happen. If you’ve never set up a Raspberry Pi before, we recommend the . official Raspberry Pi setup guide Connect your Device to the Cloud After this next step is completed, the Raspberry Pis will be able to connect to the internet regardless of whether or not it has a WiFi connection. This means that we’ll be able to send commands to and receive data from the electric scooter so that it can be used anywhere in the world. It also helps eliminate connection loss issues that might make it difficult for us to monitor the eScooter. We’ll do this by setting up the Raspberry Pi for use with the included Soracom SIM card. We’ll utilize Soracom’s SIM card offering called , which is Soracom’s solution to connecting IoT devices to the internet. It acts as the gateway to the Soracom platform, but unlike many cellular IoT providers and services, Air also works with network providers and carriers around the world to provide global coverage. SORACOM Air The following is adapted from Soracom’s official documentation here . To use the SORACOM Air SIM card with your Raspberry Pi, you’ll first need to add it to your Soracom account. Do this by visiting the and clicking “register SIM”. If you’ve added the card successfully, you should see the page refresh with a new offline entry in the table. user console Next, you’ll need to add some dependencies. Open up a terminal on the Raspberry Pi or SSH into it and run the following commands: $ sudo curl -O https://soracom-files.s3.amazonaws.com/starterkit/setup.sh
$ sudo chmod +x setup.sh
$ sudo ./setup.sh After this step, you should see a message asking you to plug the Huawei 3G modem in. At this point, pop out the SIM card and insert it into the modem dongle. The Huawei 3G modem’s case reveals a SIM card slot if you gently slide the top cover away from the USB port. Slide the SIM card into the modem, close it up, and connect it to the Raspberry Pi with the included cable. The modem’s status indicator should now be a solid blue if a connection was established. If you did your setup correctly, you should now see that your SIM card is online on your SORACOM console. If you’re having trouble getting the Huawei 3G modem to connect, you might need to add a new connection to the Raspberry Pi by running $ sudo nmcli con gsm ifname con-name soracom apn soracom.io sora password sora add type "*" user and rebooting the Raspberry Pi with $ sudo reboot Your Raspberry Pi is now connected to SORACOM Air! Not only can you use SORACOM Air to access private and public services, but you can also use Air to track each IoT device’s data consumption, cellular location, and more. Congrats! Connect the GPS Module On the Soracom dashboard, there is a map indicating where the SIM card thinks it is. If SIM’s connection to their cloud is anything like ours, you’ll find that the estimate is not nearly accurate enough for locating an electric scooter. Therefore, we’ll need to attach an additional GPS module to the Raspberry Pi to keep track of exactly where the eScooter is. Of course, with the now-connected SIM card, we’ll be able to access this information from anywhere through the cloud. We found this inexpensive GPS module called the . It seems to work surprisingly well. Before we start wiring the module up, let’s configure the Raspberry Pi to use hardware serial connections. Beitian BN-180 on Amazon Type $ sudo raspi-config Navigate to and press Interfacing Options ENTER Go to and press Serial ENTER Disable the login shell by selecting and press No ENTER Enable the hardware serial by selecting and press Yes ENTER Reboot the Raspberry Pi. You’ll want to connect the wire to 3.3v, the to ground, the to TX, and finally to RX. If your Raspberry Pi is powered on, the GPS module should now blink blue. If it’s flashing red as well, that means the module has locked onto the GPS satellites and is receiving data. We were able to get it blinking red indoors, but you may need to place the antenna on a windowsill or set the device up outside for it to connect. red black green white If you’re using SSH and having trouble connecting after powering on the module, try disconnecting the GPS module, retrying SSH, and then reconnecting the GPS module. To test our GPS module, we simply need to install a couple of packages. Open a terminal or SSH again and run $ sudo apt install gpsd gpsd-clients After that’s finished installing, let’s run the GPSD client $ sudo gpsd ttyS0 -F run /dev/ /var/ /gpsd.sock For my Raspberry Pi, ` ` is the serial port that receives data from the GPS module. Let’s check the connection /dev/ttyS0 $ cat /dev/ttyS0 If this logs out a continuous stream of data, the GPS module is working! This is raw GPS data in NMEA format. If you see a bunch of messages in the log, try this command to silence those messages unknown $ stty -F /dev/ttyS0 - echo Finally, let’s output this data into a nice format $ cgps -s The command may take a while for data to show up. If the command shows data while the command does not, edit the file. Change the line cat cgps /etc/default/gpsd =”” DEVICES to include your serial port. For example, ours became =”/dev/ttyS0” DEVICES And restart the GPSD service $ sudo systemctl restart gpsd Now rerun the command $ cgps -s The GPS module was set up successfully. Great! Connecting your IoT Device to the eScooter Now that you have a Raspberry Pi ready to go with Soracom, let’s set it up for the electric scooter. We’ll finally connect our Raspberry Pi to the eScooter via the Xiaomi M365’s Bluetooth interface. By doing so, we’ll have access to trip and battery data, as well as switches for locking the eScooter. This makes managing the electric scooter programmatically and remotely much easier. Thankfully, most of the required to communicate with the device has been completed. We just have to link the functions from the Python module to the Python controller. reverse engineering I found this Xiaomi M365 Bluetooth module on GitHub by Anton Hakansson. This module will work great for our project. Let’s set up our Raspberry Pi to use this module $ sudo apt install libglib2.0-dev python3-pip git vim -y Before we can install the module, we need to install its dependency, Bluepy $ sudo pip3 install bluepy Let’s install the library $ sudo pip3 install git+https: thub.com 365py.git //gi /AntonHakansson/m #egg=m365py And let’s download some examples to test out the module $ git ://github.com/AntonHakansson/m365py clone https Before we can run the example programs, we need to get the eScooter’s Bluetooth MAC address. Luckily, the library provides a function that lets us do just that. Turn on the electric scooter and run $ sudo python3 -m m365py After five seconds, it should output an object with a MAC address. Copy this MAC address. Now in the cloned repository, into the directory. Edit the file and change the contents of the variable on line 30 with the MAC address you copied from earlier. Save and close the file. Now run the program! cd examples main.py scooter_mac_address $ python main.py Great! If all went well, the electric scooter’s Bluetooth data should be logging to the screen. Conclusion Once you’ve completed the steps above, the hardest part of setting up your electric scooter company should be done!  We created a cellular connection to Soracom and a Bluetooth connection to the Xiaomi Mijia M365 electric scooter on the Raspberry Pi. We can now access the internet on Raspberry Pi, and we are now able to collect data about the eScooter using its Bluetooth interface. For us, seeing each step be completed was really exciting. We were able to see the GPS module receive location data in real-time and even while the eScooter was moving. It was cool to see the Raspberry Pi connect to the SIM card and overcome the limitation of needing to be in range of a WiFi network to access the internet. Our setup process went smoothly and took around forty minutes, including time spent waiting for packages to download. Following the commands were fairly easy as well, because many of the setup scripts were luckily bundled together by Soracom or other GitHub users. The most difficult part was working with the GPS module, which we’ve hopefully made more straightforward with our instructions above. Join us in the next article, where we’ll discuss how we can use the Soracom platform for data analytics and visualization. See you next time!

Building a Cellular-Connected IoT Electric Scooter w/ Soracom + Raspberry Pi in One Hour [Part 1]

We’ll assume that you’ve completed the steps from the first part of our eScooter series already. For broader overview of this project, check out the first article here . Now that you have your electric scooter connected to the internet with , let’s send some data to the cloud. This step will help secure the electric scooter by storing data remotely. SORACOM Air We'll send each data packet through Soracom's secure cellular network, so it’ll be incredibly difficult for a malicious hacker to tamper with the electric scooter and access its data. We’ll also be able to easily track the electric scooter’s location, battery, and speed. Normally, we’d have to create and manage our own database and server solutions to do all this, but luckily, Soracom has their own solution for managing data. We can just use to collect and store data. It’s super easy to set up too. All the Raspberry Pi has to do is send an API request to the Harvest endpoint which can be accomplished in a couple lines of code. Soracom conveniently has another service on their platform called that lets us easily visualize our data with charts, maps, and tables. We don’t even have to write any code! Lagoon can connect to Harvest automatically, so we don’t need to worry about shuffling data between different servers and services. SORACOM Harvest SORACOM Lagoon Data Collection Setup We’ll start by configuring . This step will create a place to receive and store eScooter data from the Raspberry Pi, once we set that up later in this article. We’ll then be able to access this data for visualizations. SORACOM Harvest Normally this step would consist of setting up a database and server endpoints for data entry. We would also have to worry about authentication and IoT security. However, Harvest combines and greatly simplifies this process. It provides a unified endpoint for data collection, abstracting away the need to construct your own server API. It also securely stores data in the cloud and provides authentication using the Air SIM card, preventing malicious hackers from falsifying data. All we have to do is configure SORACOM Harvest and configure our IoT device to send data to our Harvest endpoint. Open up your Soracom console and click on your group. Find the SORACOM Harvest tab and open it. Now just toggle the switch on. Once we have that set up, we’re done — no coding necessary! Push Data into the Cloud with Raspberry Pi and Python Let’s get back to our electric scooter. Now that we have endpoints for collecting data set up, we’ll need to configure the Raspberry Pi to actually use them. In this step, we’ll transition away from collecting data about each individual eScooter to aggregating data in the cloud. We’ll be able to see a broader overview of all our electric scooters, rather than manually querying each one. The script on our Raspberry Pi from the last article let us access useful eScooter information such as its battery life, current speed, and GPS coordinates through the Xiaomi Mijia M365’s Bluetooth protocols. In this section, we’ll send this information to the SORACOM Harvest cloud. To get started, let’s clone the code necessary to connect to SORACOM Harvest $ git https://github.com/DevinMui/soracom-escooter-device clone And let’s install some necessary modules $ pip3 install -r requirements.txt Inside , we send an HTTP request to the SORACOM Harvest endpoint we set up in the last step. We’ll collect device data from the device object’s properties, convert it into a JSON object through the module, and return SORACOM Harvest’s response. harvest.py POST requests Before running with Python, we’ll need to change the Bluetooth MAC address in the file to match the scooter we’re working on. This information should be available from the Bluetooth module we set up in the first article. Open up the file in a text editor and change line 7 to use your address. harvest.py Now we can run the program by calling $ python3 harvest.py In a matter of seconds, data will start pouring into your Soracom user console. Take a look! Visualize IoT Data After we have collected eScooter data, we need to be able to quickly access it. Through Harvest, we can access the raw data or simple tables and graphs, but it would be overwhelming to trudge through the data streams of each eScooter as more are added to the company or to generate more useful graphs like maps. We would instead rely on a visualization service to parse the data and return useful charts or graphs. Luckily, Soracom has provided . It connects to Harvest and offers intuitive dashboards without setting up additional infrastructure. Using Lagoon, we can quickly see which electric scooters need maintenance or a charge. The map functionality is also useful to find out exactly where each eScooter is. Lagoon also lets us set up alerts about the data it’s getting, so we can get notified when an eScooter needs a quick trip to repairs. Lagoon Let’s begin by setting up Lagoon to show every eScooter on a map. Let’s list out every eScooter and its corresponding data. Click on add a panel and table. Now in the settings, select your data source as Harvest. This will automatically pull data from Harvest. You can then select what data points you want to view in your chart. A simple list is kind of boring though so let’s make a map to visualize all the eScooters in our location. Again, click new panel. This time, click Soracom Map Panel. In the settings, select Harvest as your data source. Your map should look like this now. Now, let’s graph our electric scooter’s battery usage. Again, click new panel. Select Harvest as the data source in the options. Select battery as one of the data points to view. Your graph should look something like this now. We can even set up real-time alerts by email, Slack, or webhook! Let’s make it so that we get an email whenever a eScooter is running low on power. Under the graph settings panel, there should be an alert tab. Click on alert. Name your alert and change the settings however you like. We set it up to alert us when the last battery value is below 1% and set it to evaluate every 60 seconds. In the notifications tab, edit the send to value to be an email address or phone number. When the alert is triggered, you will be notified through this value. Now, when the eScooter drops to under 1% battery, we’ll get an email notification along with the electric scooter’s GPS coordinates on a map along with its current data. Conclusion You should now be able to securely stream and visualize important eScooter metrics, like position and battery. We created a SORACOM Harvest endpoint and used the networking capabilities of the Raspberry Pi to send data to the cloud. We also created a Lagoon dashboard, which provides us with useful charts and graphs generated from our data. We could see the potential uses for eScooter data as we watched the data flow in from the Raspberry Pi. We especially liked the convenience and speed of the service and thought about applications that could leverage these benefits. For example, we could set up Harvest and Lagoon to help manage a maintenance crew or reward users for charging eScooters. We could also manipulate eScooter pricing based on the data we collected and add additional electric scooters in areas with increased usage. Setting up the two services was fairly easy and took much less time than the setup process in the first article. Both Lagoon and Harvest have fairly intuitive user interfaces and it didn’t take long to configure it to do exactly what we wanted. We didn’t even need to install any packages on the Raspberry Pi. In the next article, we’ll remotely connect to and manage our scooter via SSH and HTTP. See you next time!

IoT Electric Scooter Cloud Data Collection and Visualization with Soracom + Raspberry Pi [Part 2]

Authors:
(1) Michael Sorochan Armstrong, Computational Data Science (CoDaS) Lab in the Department of Signal Theory, Telematics, and Communications at the University of Granada;
(2) Jose Carlos P´erez-Gir´on, part of the Interuniversity Institute for Research on the Earth System in Andalucia through the University of Granada;
(3) Jos´e Camacho, Computational Data Science (CoDaS) Lab in the Department of Signal Theory, Telematics, and Communications at the University of Granada;
(4) Regino Zamora, part of the Interuniversity Institute for Research on the Earth System in Andalucia through the University of Granada. Table of Links Abstract & Introduction Optimization of the Optical Interpolation Materials and Methods Results and Discussion Conclusion Appendix A: Proof of Hermitian Self-Adjoint product identity for Equidistant Time-Domain Measurements Appendix B: AAH ̸= MIN I in the Non-Equidistant Case Acknowledgments & References ACKNOWLEDGMENTS The proof for the equidistant case was adapted from a proof by David J. Fleet and Allan D. Jepson from the University of Toronto [18]. REFERENCES [1] C. Haley, “Missing-data nonparametric coherency estimation,” IEEE Signal Processing Letters, vol. 28, pp. 1704–1708, 2021. [2] S. Dilmaghani, I. C. Henry, P. Soonthornnonda, E. R. Christensen, and R. C. Henry, “Harmonic analysis of environmental time series with missing data or irregular sample spacing,” Environmental science & technology, vol. 41, no. 20, pp. 7030–7038, 2007. [3] J. W. Cooley and J. W. Tukey, “An algorithm for the machine calculation of complex fourier series,” Mathematics of computation, vol. 19, no. 90, pp. 297–301, 1965. [4] F. Rasheed, P. Peng, R. Alhajj, and J. Rokne, “Fourier transform based spatial outlier mining,” in Intelligent Data Engineering and Automated Learning-IDEAL 2009: 10th International Conference, Burgos, Spain, September 23-26, 2009. Proceedings 10. Springer, 2009, pp. 317–324. [5] C. Van Loan, Computational frameworks for the fast Fourier transform. SIAM, 1992. [6] H. Carslaw, “A historical note on gibbs’ phenomenon in fourier’s series and integrals,” Bulletin of the American Mathematical Society, 1925. [7] J. Markel, “Fft pruning,” IEEE transactions on Audio and Electroacoustics, vol. 19, no. 4, pp. 305–311, 1971. [8] J. C. Bowman and Z. Ghoggali, “The partial fast fourier transform,” Journal of Scientific Computing, vol. 76, pp. 1578–1593, 2018. [9] M. Kircheis and D. Potts, “Direct inversion of the nonequispaced fast fourier transform,” Linear Algebra and its Applications, vol. 575, pp. 106–140, 2019. [10] S. Kunis and D. Potts, “Stability results for scattered data interpolation by trigonometric polynomials,” SIAM Journal on Scientific Computing, vol. 29, no. 4, pp. 1403–1419, 2007. [11] K. B. Petersen, M. S. Pedersen et al., “The matrix cookbook,” Technical University of Denmark, vol. 7, no. 15, p. 510, 2008. [12] C. M. Bishop, Neural networks for pattern recognition. Oxford university press, 1995. [13] H. Wickham, M. Averick, J. Bryan, W. Chang, L. D. McGowan, R. Franc¸ois, G. Grolemund, A. Hayes, L. Henry, J. Hester, M. Kuhn, T. L. Pedersen, E. Miller, S. M. Bache, K. Muller, J. Ooms, D. Robin- ¨ son, D. P. Seidel, V. Spinu, K. Takahashi, D. Vaughan, C. Wilke, K. Woo, and H. Yutani, “Welcome to the tidyverse,” Journal of Open Source Software, vol. 4, no. 43, p. 1686, 2019. [14] R Core Team, R: A Language and Environment for Statistical Computing, R Foundation for Statistical Computing, Vienna, Austria, 2023. [Online]. Available: https://www.R-project.org/ [15] C. R. Harris, K. J. Millman, S. J. van der Walt, R. Gommers, P. Virtanen, D. Cournapeau, E. Wieser, J. Taylor, S. Berg, N. J. Smith, R. Kern, M. Picus, S. Hoyer, M. H. van Kerkwijk, M. Brett, A. Haldane, J. F. del R´ıo, M. Wiebe, P. Peterson, P. Gerard-Marchant, K. Sheppard, T. Reddy, W. Weckesser, ´ H. Abbasi, C. Gohlke, and T. E. Oliphant, “Array programming with NumPy,” Nature, vol. 585, no. 7825, pp. 357–362, Sep. 2020. [Online]. Available: https://doi.org/10.1038/s41586-020-2649-2 [16] P. Virtanen, R. Gommers, T. E. Oliphant, M. Haberland, T. Reddy, D. Cournapeau, E. Burovski, P. Peterson, W. Weckesser, J. Bright, S. J. van der Walt, M. Brett, J. Wilson, K. J. Millman, N. Mayorov, A. R. J. Nelson, E. Jones, R. Kern, E. Larson, C. J. Carey, ˙I. Polat, Y. Feng, E. W. Moore, J. VanderPlas, D. Laxalde, J. Perktold, R. Cimrman, I. Henriksen, E. A. Quintero, C. R. Harris, A. M. Archibald, A. H. Ribeiro, F. Pedregosa, P. van Mulbregt, and SciPy 1.0 Contributors, “SciPy 1.0: Fundamental Algorithms for Scientific Computing in Python,” Nature Methods, vol. 17, pp. 261– 272, 2020. [17] J. D. Hunter, “Matplotlib: A 2d graphics environment,” Computing in Science & Engineering, vol. 9, no. 3, pp. 90–95, 2007. [18] “Notes on fourier analysis,” http://www.cs.toronto.edu/ ∼jepson/csc320/notes/fourier.pdf, accessed: 2023-09-01. This paper is available on arxiv under CC 4.0 license. Authors: (1) Michael Sorochan Armstrong, Computational Data Science (CoDaS) Lab in the Department of Signal Theory, Telematics, and Communications at the University of Granada; (2) Jose Carlos P´erez-Gir´on, part of the Interuniversity Institute for Research on the Earth System in Andalucia through the University of Granada; (3) Jos´e Camacho, Computational Data Science (CoDaS) Lab in the Department of Signal Theory, Telematics, and Communications at the University of Granada; (4) Regino Zamora, part of the Interuniversity Institute for Research on the Earth System in Andalucia through the University of Granada. Authors: Authors: (1) Michael Sorochan Armstrong, Computational Data Science (CoDaS) Lab in the Department of Signal Theory, Telematics, and Communications at the University of Granada; (2) Jose Carlos P´erez-Gir´on, part of the Interuniversity Institute for Research on the Earth System in Andalucia through the University of Granada; (3) Jos´e Camacho, Computational Data Science (CoDaS) Lab in the Department of Signal Theory, Telematics, and Communications at the University of Granada; (4) Regino Zamora, part of the Interuniversity Institute for Research on the Earth System in Andalucia through the University of Granada. Table of Links Abstract & Introduction Abstract & Introduction Optimization of the Optical Interpolation Optimization of the Optical Interpolation Materials and Methods Materials and Methods Results and Discussion Results and Discussion Conclusion Conclusion Appendix A: Proof of Hermitian Self-Adjoint product identity for Equidistant Time-Domain Measurements Appendix A: Proof of Hermitian Self-Adjoint product identity for Equidistant Time-Domain Measurements Appendix B: AAH ̸= MIN I in the Non-Equidistant Case Appendix B: AAH ̸= MIN I in the Non-Equidistant Case Acknowledgments & References Acknowledgments & References ACKNOWLEDGMENTS The proof for the equidistant case was adapted from a proof by David J. Fleet and Allan D. Jepson from the University of Toronto [18]. REFERENCES [1] C. Haley, “Missing-data nonparametric coherency estimation,” IEEE Signal Processing Letters, vol. 28, pp. 1704–1708, 2021. [2] S. Dilmaghani, I. C. Henry, P. Soonthornnonda, E. R. Christensen, and R. C. Henry, “Harmonic analysis of environmental time series with missing data or irregular sample spacing,” Environmental science & technology, vol. 41, no. 20, pp. 7030–7038, 2007. [3] J. W. Cooley and J. W. Tukey, “An algorithm for the machine calculation of complex fourier series,” Mathematics of computation, vol. 19, no. 90, pp. 297–301, 1965. [4] F. Rasheed, P. Peng, R. Alhajj, and J. Rokne, “Fourier transform based spatial outlier mining,” in Intelligent Data Engineering and Automated Learning-IDEAL 2009: 10th International Conference, Burgos, Spain, September 23-26, 2009. Proceedings 10. Springer, 2009, pp. 317–324. [5] C. Van Loan, Computational frameworks for the fast Fourier transform. SIAM, 1992. [6] H. Carslaw, “A historical note on gibbs’ phenomenon in fourier’s series and integrals,” Bulletin of the American Mathematical Society, 1925. [7] J. Markel, “Fft pruning,” IEEE transactions on Audio and Electroacoustics, vol. 19, no. 4, pp. 305–311, 1971. [8] J. C. Bowman and Z. Ghoggali, “The partial fast fourier transform,” Journal of Scientific Computing, vol. 76, pp. 1578–1593, 2018. [9] M. Kircheis and D. Potts, “Direct inversion of the nonequispaced fast fourier transform,” Linear Algebra and its Applications, vol. 575, pp. 106–140, 2019. [10] S. Kunis and D. Potts, “Stability results for scattered data interpolation by trigonometric polynomials,” SIAM Journal on Scientific Computing, vol. 29, no. 4, pp. 1403–1419, 2007. [11] K. B. Petersen, M. S. Pedersen et al., “The matrix cookbook,” Technical University of Denmark, vol. 7, no. 15, p. 510, 2008. [12] C. M. Bishop, Neural networks for pattern recognition. Oxford university press, 1995. [13] H. Wickham, M. Averick, J. Bryan, W. Chang, L. D. McGowan, R. Franc¸ois, G. Grolemund, A. Hayes, L. Henry, J. Hester, M. Kuhn, T. L. Pedersen, E. Miller, S. M. Bache, K. Muller, J. Ooms, D. Robin- ¨ son, D. P. Seidel, V. Spinu, K. Takahashi, D. Vaughan, C. Wilke, K. Woo, and H. Yutani, “Welcome to the tidyverse,” Journal of Open Source Software, vol. 4, no. 43, p. 1686, 2019. [14] R Core Team, R: A Language and Environment for Statistical Computing, R Foundation for Statistical Computing, Vienna, Austria, 2023. [Online]. Available: https://www.R-project.org/ [15] C. R. Harris, K. J. Millman, S. J. van der Walt, R. Gommers, P. Virtanen, D. Cournapeau, E. Wieser, J. Taylor, S. Berg, N. J. Smith, R. Kern, M. Picus, S. Hoyer, M. H. van Kerkwijk, M. Brett, A. Haldane, J. F. del R´ıo, M. Wiebe, P. Peterson, P. Gerard-Marchant, K. Sheppard, T. Reddy, W. Weckesser, ´ H. Abbasi, C. Gohlke, and T. E. Oliphant, “Array programming with NumPy,” Nature, vol. 585, no. 7825, pp. 357–362, Sep. 2020. [Online]. Available: https://doi.org/10.1038/s41586-020-2649-2 [16] P. Virtanen, R. Gommers, T. E. Oliphant, M. Haberland, T. Reddy, D. Cournapeau, E. Burovski, P. Peterson, W. Weckesser, J. Bright, S. J. van der Walt, M. Brett, J. Wilson, K. J. Millman, N. Mayorov, A. R. J. Nelson, E. Jones, R. Kern, E. Larson, C. J. Carey, ˙I. Polat, Y. Feng, E. W. Moore, J. VanderPlas, D. Laxalde, J. Perktold, R. Cimrman, I. Henriksen, E. A. Quintero, C. R. Harris, A. M. Archibald, A. H. Ribeiro, F. Pedregosa, P. van Mulbregt, and SciPy 1.0 Contributors, “SciPy 1.0: Fundamental Algorithms for Scientific Computing in Python,” Nature Methods, vol. 17, pp. 261– 272, 2020. [17] J. D. Hunter, “Matplotlib: A 2d graphics environment,” Computing in Science & Engineering, vol. 9, no. 3, pp. 90–95, 2007. [18] “Notes on fourier analysis,” http://www.cs.toronto.edu/ ∼jepson/csc320/notes/fourier.pdf, accessed: 2023-09-01. This paper is available on arxiv under CC 4.0 license. This paper is available on arxiv under CC 4.0 license. available on arxiv

Groundbreaking Research in Signal Theory

Authors:
(1) Michael Sorochan Armstrong, Computational Data Science (CoDaS) Lab in the Department of Signal Theory, Telematics, and Communications at the University of Granada;
(2) Jose Carlos P´erez-Gir´on, part of the Interuniversity Institute for Research on the Earth System in Andalucia through the University of Granada;
(3) Jos´e Camacho, Computational Data Science (CoDaS) Lab in the Department of Signal Theory, Telematics, and Communications at the University of Granada;
(4) Regino Zamora, part of the Interuniversity Institute for Research on the Earth System in Andalucia through the University of Granada. Table of Links Abstract & Introduction Optimization of the Optical Interpolation Materials and Methods Results and Discussion Conclusion Appendix A: Proof of Hermitian Self-Adjoint product identity for Equidistant Time-Domain Measurements Appendix B: AAH ̸= MIN I in the Non-Equidistant Case Acknowledgments & References APPENDIX A: PROOF OF HERMITIAN SELF-ADJOINT PRODUCT IDENTITY FOR EQUIDISTANT TIME-DOMAIN MEASUREMENTS Consider the case of an NFFT matrix and its hermitian, where: Equation 44 simplifies to: This paper is available on arxiv under CC 4.0 license. Authors: (1) Michael Sorochan Armstrong, Computational Data Science (CoDaS) Lab in the Department of Signal Theory, Telematics, and Communications at the University of Granada; (2) Jose Carlos P´erez-Gir´on, part of the Interuniversity Institute for Research on the Earth System in Andalucia through the University of Granada; (3) Jos´e Camacho, Computational Data Science (CoDaS) Lab in the Department of Signal Theory, Telematics, and Communications at the University of Granada; (4) Regino Zamora, part of the Interuniversity Institute for Research on the Earth System in Andalucia through the University of Granada. Authors: Authors: (1) Michael Sorochan Armstrong, Computational Data Science (CoDaS) Lab in the Department of Signal Theory, Telematics, and Communications at the University of Granada; (2) Jose Carlos P´erez-Gir´on, part of the Interuniversity Institute for Research on the Earth System in Andalucia through the University of Granada; (3) Jos´e Camacho, Computational Data Science (CoDaS) Lab in the Department of Signal Theory, Telematics, and Communications at the University of Granada; (4) Regino Zamora, part of the Interuniversity Institute for Research on the Earth System in Andalucia through the University of Granada. Table of Links Abstract & Introduction Abstract & Introduction Optimization of the Optical Interpolation Optimization of the Optical Interpolation Materials and Methods Materials and Methods Results and Discussion Results and Discussion Conclusion Conclusion Appendix A: Proof of Hermitian Self-Adjoint product identity for Equidistant Time-Domain Measurements Appendix A: Proof of Hermitian Self-Adjoint product identity for Equidistant Time-Domain Measurements Appendix B: AAH ̸= MIN I in the Non-Equidistant Case Appendix B: AAH ̸= MIN I in the Non-Equidistant Case Acknowledgments & References Acknowledgments & References APPENDIX A: PROOF OF HERMITIAN SELF-ADJOINT PRODUCT IDENTITY FOR EQUIDISTANT TIME-DOMAIN MEASUREMENTS Consider the case of an NFFT matrix and its hermitian, where: Equation 44 simplifies to: This paper is available on arxiv under CC 4.0 license. This paper is available on arxiv under CC 4.0 license. available on arxiv

Proof of Hermitian Self-Adjoint Product Identity for Equidistant Time-Domain Measurements

Authors: (1) Michael Sorochan Armstrong, Computational Data Science (CoDaS) Lab in the Department of Signal Theory, Telematics, and Communications at the University of Granada; (2) Jose Carlos P´erez-Gir´on, part of the Interuniversity Institute for Research on the Earth System in Andalucia through the University of Granada; (3) Jos´e Camacho, Computational Data Science (CoDaS) Lab in the Department of Signal Theory, Telematics, and Communications at the University of Granada; (4) Regino Zamora, part of the Interuniversity Institute for Research on the Earth System in Andalucia through the University of Granada. Table of Links Abstract & Introduction Optimization of the Optical Interpolation Materials and Methods Results and Discussion Conclusion Appendix A: Proof of Hermitian Self-Adjoint product identity for Equidistant Time-Domain Measurements Appendix B: AAH ̸= MIN I in the Non-Equidistant Case Acknowledgments & References since it is obvious that when θ = 0, the only possible evaluation for z is 1, consider only when θ ̸= 0, which we can write as: to evaluate for z for some constant integer θ, consider the finite geometric series: This paper is under CC 4.0 license. available on arxiv

Exploring Advanced Time-Domain Measurement Techniques

উন্নত সময়-ডোমেন পরিমাপ কৌশল অন্বেষণ

Erkundung fortschrittlicher Zeitbereichsmesstechniken

Explorando técnicas avanzadas de medición en el dominio del tiempo

Explorer les techniques avancées de mesure du domaine temporel

उन्नत समय-डोमेन मापन तकनीकों की खोज

高度な時間領域測定技術の探索

고급 시간 영역 측정 기술 탐색

Explorando técnicas avançadas de medição no domínio do tempo

Изучение передовых методов измерения во временной области

Gelişmiş Zaman Alanı Ölçüm Tekniklerini Keşfetmek

Khám phá các kỹ thuật đo lường miền thời gian nâng cao

探索先进的时域测量技术

Authors:
(1) Michael Sorochan Armstrong, Computational Data Science (CoDaS) Lab in the Department of Signal Theory, Telematics, and Communications at the University of Granada;
(2) Jose Carlos P´erez-Gir´on, part of the Interuniversity Institute for Research on the Earth System in Andalucia through the University of Granada;
(3) Jos´e Camacho, Computational Data Science (CoDaS) Lab in the Department of Signal Theory, Telematics, and Communications at the University of Granada;
(4) Regino Zamora, part of the Interuniversity Institute for Research on the Earth System in Andalucia through the University of Granada. Table of Links Abstract & Introduction Optimization of the Optical Interpolation Materials and Methods Results and Discussion Conclusion Appendix A: Proof of Hermitian Self-Adjoint product identity for Equidistant Time-Domain Measurements Appendix B: AAH ̸= MIN I in the Non-Equidistant Case Acknowledgments & References 5 CONCLUSION A convenient mathematical expression to solve for the spectrum of irregularly sampled time-domain data has been demonstrated in a way that satisfies the conditions for an inverse non-uniform Fourier transform. This methodology has been derived from cost function proposed in the original study, and via its idealised quadratic form strongly suggests that its minimum is unique and convex. Using a randomly sample time series from data with few missing values, the observed relative error is on the same order of magnitude as the most current solution to this problem [10]. Upon cross-validation with contiguous blocks, we report a relative error that is much higher, and suggest this is a more critical, but informative lens through which to view interpolation problems. he proposed formula will find use in a variety of signal processing applications, where the use of an FFT is impractical given its assumption of a consistent time-domain density. The software package used in this work is available online: https://github.com/CoDaSLab/intrp infft 1d This paper is available on arxiv under CC 4.0 license. Authors: (1) Michael Sorochan Armstrong, Computational Data Science (CoDaS) Lab in the Department of Signal Theory, Telematics, and Communications at the University of Granada; (2) Jose Carlos P´erez-Gir´on, part of the Interuniversity Institute for Research on the Earth System in Andalucia through the University of Granada; (3) Jos´e Camacho, Computational Data Science (CoDaS) Lab in the Department of Signal Theory, Telematics, and Communications at the University of Granada; (4) Regino Zamora, part of the Interuniversity Institute for Research on the Earth System in Andalucia through the University of Granada. Authors: Authors: (1) Michael Sorochan Armstrong, Computational Data Science (CoDaS) Lab in the Department of Signal Theory, Telematics, and Communications at the University of Granada; (2) Jose Carlos P´erez-Gir´on, part of the Interuniversity Institute for Research on the Earth System in Andalucia through the University of Granada; (3) Jos´e Camacho, Computational Data Science (CoDaS) Lab in the Department of Signal Theory, Telematics, and Communications at the University of Granada; (4) Regino Zamora, part of the Interuniversity Institute for Research on the Earth System in Andalucia through the University of Granada. Table of Links Abstract & Introduction Abstract & Introduction Optimization of the Optical Interpolation Optimization of the Optical Interpolation Materials and Methods Materials and Methods Results and Discussion Results and Discussion Conclusion Conclusion Appendix A: Proof of Hermitian Self-Adjoint product identity for Equidistant Time-Domain Measurements Appendix A: Proof of Hermitian Self-Adjoint product identity for Equidistant Time-Domain Measurements Appendix B: AAH ̸= MIN I in the Non-Equidistant Case Appendix B: AAH ̸= MIN I in the Non-Equidistant Case Acknowledgments & References Acknowledgments & References 5 CONCLUSION A convenient mathematical expression to solve for the spectrum of irregularly sampled time-domain data has been demonstrated in a way that satisfies the conditions for an inverse non-uniform Fourier transform. This methodology has been derived from cost function proposed in the original study, and via its idealised quadratic form strongly suggests that its minimum is unique and convex. Using a randomly sample time series from data with few missing values, the observed relative error is on the same order of magnitude as the most current solution to this problem [10]. Upon cross-validation with contiguous blocks, we report a relative error that is much higher, and suggest this is a more critical, but informative lens through which to view interpolation problems. he proposed formula will find use in a variety of signal processing applications, where the use of an FFT is impractical given its assumption of a consistent time-domain density. The software package used in this work is available online: https://github.com/CoDaSLab/intrp infft 1d This paper is available on arxiv under CC 4.0 license. This paper is available on arxiv under CC 4.0 license. available on arxiv

Novel Mathematical Expression for Spectrum Determination of Irregularly Sampled Data

Authors: (1) Michael Sorochan Armstrong, Computational Data Science (CoDaS) Lab in the Department of Signal Theory, Telematics, and Communications at the University of Granada; (2) Jose Carlos P´erez-Gir´on, part of the Interuniversity Institute for Research on the Earth System in Andalucia through the University of Granada; (3) Jos´e Camacho, Computational Data Science (CoDaS) Lab in the Department of Signal Theory, Telematics, and Communications at the University of Granada; (4) Regino Zamora, part of the Interuniversity Institute for Research on the Earth System in Andalucia through the University of Granada. Table of Links Abstract & Introduction Optimization of the Optical Interpolation Materials and Methods Results and Discussion Conclusion Appendix A: Proof of Hermitian Self-Adjoint product identity for Equidistant Time-Domain Measurements Appendix B: AAH ̸= MIN I in the Non-Equidistant Case Acknowledgments & References 4 RESULTS AND DISCUSSION All training data were mean-centred prior to analysis; and the means were added to the reconstructed data prior to recording the residuals. The results summarise cross validation routines for randomly sampled data in Table 4, and randomly sampled contiguous blocks in Table 4 at 3 levels of missing data (10, 20, and 30%) performed for a total of 7 times to get the average error, and the uncertainty, as one standard deviation associated with the error measurement. Both relative error for the predicted set, and the correlation coefficient for the reconstructed data on the full data set, versus the partially sampled dataset were also performed to intuit the error due to the constraint of the reconstruction (via the weighted kernel approximation) versus the sparsity of the observed data. The relative error for the randomly sampled data is on the same order of magnitude as reported in the original study for the iterative solution to the optimal interpolation problem. However, it is clear that a more critical metric for determining the accuracy of the interpolation is by removing contiguous blocks of data, and comparing the reconstruction against the true values for those data. Despite this, and considering that the interpolation was “smoothed” using a rather aggressive low-pass filter, the results are reasonable and corroborate that the calculated spectrum of the data is relatively consistent with what data is observed, which is consistent with what we would expect a true inverse transform to return. The results also suggest that the interpolative algorithm performs significantly better relative to the equivalent truncated inverse Fast Fourier Transform (iFFT) using an equivalent number of coefficients and weight function in virtually all cases. The differences are most differentiated relative to correlation with the “true” underlying trigonometric polynomial, as shown in Tables 4 and Tables 4; only in the case of randomly sampled data is the difference inconsequential (Table 4). Despite poorer evidence of significance, on average the iNFFT algorithm in the most challenging use cases for up to 30% missing data. This paper is under CC 4.0 license. available on arxiv

Interpolative Algorithms vs. Truncated Inverse FFT: A Comparative Study

Authors:
(1) Michael Sorochan Armstrong, Computational Data Science (CoDaS) Lab in the Department of Signal Theory, Telematics, and Communications at the University of Granada;
(2) Jose Carlos P´erez-Gir´on, part of the Interuniversity Institute for Research on the Earth System in Andalucia through the University of Granada;
(3) Jos´e Camacho, Computational Data Science (CoDaS) Lab in the Department of Signal Theory, Telematics, and Communications at the University of Granada;
(4) Regino Zamora, part of the Interuniversity Institute for Research on the Earth System in Andalucia through the University of Granada. Table of Links Abstract & Introduction Optimization of the Optical Interpolation Materials and Methods Results and Discussion Conclusion Appendix A: Proof of Hermitian Self-Adjoint product identity for Equidistant Time-Domain Measurements Appendix B: AAH ̸= MIN I in the Non-Equidistant Case Acknowledgments & References 3 MATERIALS AND METHODS 3.0.1 Remote Sensor Data Soil temperature data were collected in the Sierra Nevada National Park in Andaluc´ıa, Spain, as part of the Smart EcoMountains program through the Lifewatch ERIC Program, from solar-powered remote multi-parametric micro-stations located within different environments (shrub, rock, and bare-earth) and altitudes (from 1500m to 2300m) from 29- 07-2020 until 04-05-2023. Measurements were continuously collected at a nominal 10 minute interval with micro-sensors (ibuttons, thermochron-type; Maxim integrated) connected to a network that transmits the collected information when connectivity is available. All the collected data were visually inspected for extreme values outside the normal ranges of the variable in the time series due to sensor failures, identifying a single sensor reporting erroneous values that were reported as NaNs. 3.0.2 Computational information All calculations were run with a B550 AORUS ELITE AX V2 motherboard with 128 GiB DDR4 RAM, an AMD Ryzen 9 5950x 32 core processor with an NVIDIA GeForce RTX 3080 graphics processor, and a 2 TB NVMe Samsung SSD 980. Software was installed on Ubuntu 22.04.3 LTS 64-bit operating system “Jammy Jellyfish”. Graphical inspections and data cleaning were conducted with the tidyverse 2.0.0 package [13] implemented in the R 4.3.1 software environment [14]. Calculations were performed using Python 3.10. Data was parsed and cleaned from the original .csv values using Pandas 2.0.3. Various numerical operations were performed using the vectorized functions available in NumPy 1.25 [15], with additional statistical functionality from SciPy 1.11 [16]. Graphics were created using Matplotlib 3.7.1 [17]. This paper is available on arxiv under CC 4.0 license. Authors: (1) Michael Sorochan Armstrong, Computational Data Science (CoDaS) Lab in the Department of Signal Theory, Telematics, and Communications at the University of Granada; (2) Jose Carlos P´erez-Gir´on, part of the Interuniversity Institute for Research on the Earth System in Andalucia through the University of Granada; (3) Jos´e Camacho, Computational Data Science (CoDaS) Lab in the Department of Signal Theory, Telematics, and Communications at the University of Granada; (4) Regino Zamora, part of the Interuniversity Institute for Research on the Earth System in Andalucia through the University of Granada. Authors: Authors: (1) Michael Sorochan Armstrong, Computational Data Science (CoDaS) Lab in the Department of Signal Theory, Telematics, and Communications at the University of Granada; (2) Jose Carlos P´erez-Gir´on, part of the Interuniversity Institute for Research on the Earth System in Andalucia through the University of Granada; (3) Jos´e Camacho, Computational Data Science (CoDaS) Lab in the Department of Signal Theory, Telematics, and Communications at the University of Granada; (4) Regino Zamora, part of the Interuniversity Institute for Research on the Earth System in Andalucia through the University of Granada. Table of Links Abstract & Introduction Abstract & Introduction Optimization of the Optical Interpolation Optimization of the Optical Interpolation Materials and Methods Materials and Methods Results and Discussion Results and Discussion Conclusion Conclusion Appendix A: Proof of Hermitian Self-Adjoint product identity for Equidistant Time-Domain Measurements Appendix A: Proof of Hermitian Self-Adjoint product identity for Equidistant Time-Domain Measurements Appendix B: AAH ̸= MIN I in the Non-Equidistant Case Appendix B: AAH ̸= MIN I in the Non-Equidistant Case Acknowledgments & References Acknowledgments & References 3 MATERIALS AND METHODS 3.0.1 Remote Sensor Data Soil temperature data were collected in the Sierra Nevada National Park in Andaluc´ıa, Spain, as part of the Smart EcoMountains program through the Lifewatch ERIC Program, from solar-powered remote multi-parametric micro-stations located within different environments (shrub, rock, and bare-earth) and altitudes (from 1500m to 2300m) from 29- 07-2020 until 04-05-2023. Measurements were continuously collected at a nominal 10 minute interval with micro-sensors (ibuttons, thermochron-type; Maxim integrated) connected to a network that transmits the collected information when connectivity is available. All the collected data were visually inspected for extreme values outside the normal ranges of the variable in the time series due to sensor failures, identifying a single sensor reporting erroneous values that were reported as NaNs. 3.0.2 Computational information All calculations were run with a B550 AORUS ELITE AX V2 motherboard with 128 GiB DDR4 RAM, an AMD Ryzen 9 5950x 32 core processor with an NVIDIA GeForce RTX 3080 graphics processor, and a 2 TB NVMe Samsung SSD 980. Software was installed on Ubuntu 22.04.3 LTS 64-bit operating system “Jammy Jellyfish”. Graphical inspections and data cleaning were conducted with the tidyverse 2.0.0 package [13] implemented in the R 4.3.1 software environment [14]. Calculations were performed using Python 3.10. Data was parsed and cleaned from the original .csv values using Pandas 2.0.3. Various numerical operations were performed using the vectorized functions available in NumPy 1.25 [15], with additional statistical functionality from SciPy 1.11 [16]. Graphics were created using Matplotlib 3.7.1 [17]. This paper is available on arxiv under CC 4.0 license. This paper is available on arxiv under CC 4.0 license. available on arxiv

Stories, News & Mentions about SOROCO

Soroco’s Stories on HackerNoon