GreenFrameis in the Air Pollution Management Tech Industry

GreenFrame

greenframe

Today's post serves the purpose of explaining , the very feature which made it possible to receive the call audio while still being able to send audio for playback . Twilio's recent bidirectional streaming feature asynchronously This was not possible before, when communication had to be unidirectional: either you could receive the voice audio but could not respond to it, or you could send audio to the call but not get anything back. Let's get right into the tutorial. One needs to understand three things if he or she hopes to use Twilio in conjunction with her own Voice Assistant (abbreviated VA). 1. How the backend of the VA receives the user's voice and returns audio responses (HTTP or Websocket? Streaming or one-off? In which audio format?) 2. How to serve calls using a node.js server (How can a phone call to a twilio number perform a request to a Node server?) 3. How to open and use a bidirectional communication channel between node.js code and the actual phone call (Through which protocol?) We will tackle these questions one at a time. But first, let's see the prerequisites: * Some experience with node.js HTTP servers and websockets * A public server with an open port * SSL keys for the domain of the server * A Twilio account with a Twilio number, either toll-free or house number Now, with that said, let's move on to the tutorial! The VA backend Most Voice Assistants work in turns. That is, they receive a user utterance (a user phrase), they generate the answer as voice audio, they send it back to the user and then wait for the next utterance, which starts the next dialogue turn. The user cannot speak again before getting the response from the VA. So, we mainly have two data flows, the voice data from the client to the VA (flow A) and the synthesized voice response from the VA (flow B). This turn-taking model is illustrated in the following graphic: The next thing to know about our presupposed VA model, is that flow A is an audio whereas flow B is sent as a single message. This follows from the way audio is generated: user voice audio is produced sequentially and can thus be streamed. On the other hand, the VA's response synthesized voice is generated totally, not partially - the first second is available as soon as the whole message is available. stream, Finally, turn taking is managed using two signals. The first one, is sent by the client at the beginning of the user's turn (start of flow A). The other signal is It is sent by the VA backend as soon as it recognizes the end of single phrase in the incoming audio (alternatively this can be controlled by the client as well). The SINGLE_UTTERANCE_END signal is sent to the client to prevent it from sending further audio. The process can be seen in detail in the flow diagram below: RESTART_COMMUNICATION, SINGLE_UTTERANCE_END. Audio encodings, what a mess! When I first tried to connect our team's VA to Twilio, one of the first things I checked was Twilio's audio format. I had hoped for an option for wav or mp3 encodings. Alas! The only supported format is that mysterious mulaw (or u-law or μ-law) encoding, which I learned that is aimed specifically for speech audio. I also learned that it is not widely used outside of telephony; chances are that in your system you won't have mulaw, which means that you will have to do some audio conversion. Although this might take a significant amount of work, I will not cover this part. I will only mention that for python, you can convert to by using the module in this code snippet. wav mulaw audioop wave.open(wavfile, ) wav:                                                                                              
    # load frames                                                                                                                  
    raw_wav= wav.readframes(wav.getnframes())                                                                                      
    # downsample                                                                                                                   
    raw_wav_8khz, st = audioop.ratecv(raw_wav, , , , ,None)                                                                 
    # to explain the above:                                                                                                        
    # : sample depth bytes                                                                                                    
    # : number channels                                                                                                       
    # : samplerate                                                                                                           
    # : desired samplerate                                                                                                    
                                                                                                                                   
    # convert to mulaw                                                                                                             
    raw_ulaw = audioop.lin2ulaw(raw_wav_8khz,wav.getsampwidth()) with "rb" as 2 1 24000 8000 2 in 1 of 24000 8000 Twilio node.js server From this chapter onwards, we will be discussing the Twilio server itself. To insert programming logic into your voice call, you have to use a node server. Below you can see the project structure of the tutorial. This is a node project and the server is its entry point. There are some xml files with the server's TwiML resposes (don't worry if you don't know what that is) in the templates directory. Also, there is a keys folder, with the server HTTPS keys. Last, but not least, the npm dependencies are: * httpdispatcher * websocket If you are experienced with node.js, you can skip to the last part of the article ("putting it all together"). Taking it one step at a time: part 1 - TwiML without Streams Before understanding what the node server does, let's create a simple Twilio server and connect it to a Twilio number in a baby example. This baby example will later grow into our fully functional Twilio front-end for our VA! Every node.js server for Twilio has an endpoint returning some TwiML. In our case, the server will have an endpoint on path accepting HTTP POST, responding with the the contents of file , i.e.: /twiml templates/babyapp.xml This is your Voice Assistant speaking! < > Response < > Say </ > Say </ > Response What does the TwiML do? Simply it informs Twilio on how to respond to the phone call. In this case, it will merely say "This is your Voice Assistant speaking!" with a synthesized voice, before hanging up. This is the essential file for Twilio. So, before you continue, make sure you have this file.  Also, you need to make sure the dependencies are installed (as mentioned above, and ). Finally, it is necessary to spin up a server to serve this TwiML. The code responsible for sharing the TwiML is as follows: httpdispatcher websocket Store the code in . The server's work is done in 's method: is read and then sent back to the client as HTTP POST reponse body. Now you have to install the dependencies (npm-install) and run the server: twilio_server.js dispatcher onPost templates/babyapp.xml node twilio_server.js Now you should be able to access the TwiML in using curl: app.xml curl -d -X https://yourhost:1312/twiml This should return: This is your Voice Assistant speaking! < > Response < > Say </ > Say </ > Response To make this work one final step is needed. First, install the twilio-cli and setup your twilio credentials. Then, connect the endpoint of the server you just made with your Twilio number: twilio phone-numbers:update +111234567890 --voice-url http://your.host:1314/twiml Now, if you have configured your Twilio number correctly, you should be able to call it and listen to the message you have specified with the tag! <Say> Taking it one step at a time: part 2 - pipe them messages! Now it's time to change the TwiML to allow bidirectional communication with the call: This is your Voice Assistant speaking! <?xml version="1.0" encoding="UTF-8" ?> < > Response < > Say </ > Say < > Connect < = > Stream url "wss:your.host:1314/socket" </ > Stream </ > Connect </ > Response Save the new TwiML in a new file named , inside the dir. app.xml templates What is this new sorcery? In simple terms, is used to set the websocket to which the call will connect and is used to make this websocket bidirectional. The latter tag is the one of the additions made in July 2020, which made it easy to use Twilio with VA's. <Stream> <Connect> As you can see, the websocket address has the same host and port as before. The reason for this is that we will handle this connection through the same server that we use for serving the TwiML. We will call this websocket the . You need one more, one for communicating with the VA. We will call this websocket the . In the schema below, note the role of the the node server in the middle: Twilio websocket VA websocket In essence, the Twilio server is just middleware between the Twilio cloud service (which sends voice audio and receives what should be played back to the human on the phone) and our VA. It is, at the same time, a server for twilio to connect to, and a VA client. Both websockets are mostly described by their message handler; they do not perform any special actions when closed, neither do they do something special when opened, apart from binding the message handlers. Let's jump right into the code that handles incoming Twilio messages: Twilio's messages are named events, and there are 5 types of events: , , and . They are described in detail in . and events are just one-off events sent at the beginning and the ending of the call. connected start, media mark close Twilio's docs Connected close The event only occurs once too. When using the tag, Τwilio sends voice audio in base64 encoded chunks, proceeded by a single message. This message is handy for getting the of the call (a unique ID used when sending stuff to Twilio) and, of course, it signifies the start of the voice stream. start <Stream> start start streamSID This is a good time to start the communication with the VA using the signal. Also it is necessary to turn the attribute to . This attribute has two values: or . Mode is during flow A is being sent,  while mode is for the rest of the time. "Waiting" is not a really good name though, as there is stuff going on in the waiting phase too, as we will see in the next section. RESTART_COMMUNICATION mode "recording" "recording" "waiting" "recording" "waiting" The event is sent many times: it contains chunks of , recorded from the user's phone. This kind of message should be sent to the VA. Also, any necessary preprocessing should happen here. e.g changing the encoding. media streaming voice audio Lastly, the event carries a message. In Twilio it is often used for notifications. In this case, it is received from the Twilio call when playback of a VA's response has ended (if you're feeling confused, things will clear out when you read the next section). Here, a new conversation turn starts - the user can speak again. Attribute is set back to and the signal is sent to the VA. mark this.mode "recording" RESTART_COMMUNICATION Messages on the VA websocket Next is the handler of the VA websocket: Similarly to the handler of the Twilio socket, this one accepts only string messages: it handles the messages we talked about before and the audio responses from the VA. The former are used to stop the recorded audio from the call from being sent to the VA. SINGLE_UTTERANCE_END The latter are prefixed with and are sent to Twilio, to be played back to the user. But this not enough - a mark message is also sent right after the audio. This step is important, because it tells Twilio to notify the server as soon as the sound clip that was just sent has been played to the user. Notice the field: its value is the same as the name in the mark message Twilio will send afterwards. _MULAW "name" Putting it all together Now it's time to unify things into one big working server. The suggested solution is to create a class named , and create an instance of this class for every incoming Twilio connection. The two websockets we already described are attributes of . The setting up of the sockets, i.e. the connection of the client to the VA and the binding of onMessage(), onOpen(), onError() etc is done in the class constructor. MediaStream MediaStream Some details of importance: 1. The tag only supports wss (the secure websocket protocol). That means that you must have an HTTPS server with the keys ready in a separate keys folder (lines 1-15). <Stream> By the way, the options object used to create the server also contains the path for the Twilio websocket endpoint provided in the TwiML. This HTTPS server will be fed into the constructor of the WebsocketServer object. The latter will listen to the endpoint specified in the TwiML (lines 16-20). 2. The code in lines 26-47 is the same as the first part, it is made for serving the TwiML inside the .xml. 3. The next part (lines 49-81) might look a bit perplexing. Essentially, it is the code that establishes the connections with Twilio and the VA. A bit more explantion is due: Everything starts when Twilio is connected to the websocket endpoint. This causes the creation of a MediaStream object, with the connection object as an argument. The code in the constructor ensures that the processing of messages from Twilio will start the websocket with the VA has been opened. after This is achieved by binding the function to the onConnected event, in line 61: when the connection with the VA is complete, then it is safe to bind the message handlers to the websocket connections. prepareWebsockets() That's all for today. Let me know how it went, and feel free to ask any questions in the comments section. Happy coding!

Twilio: How to Connect with Existing Voice Assistant 

Galen Framework is a test automation framework which was originally introduced to perform cross browser layout testing of a web application in a browser. Nowadays, it has become a fully functional testing framework with rich reporting and test management system. This framework supports both Java and Javascript. Are you wondering about what cross browser layout testing is? Well let me clarify, so you develop a fast-functioning website using Google Chrome. Everything is working as per intended functionality for you. The UI is neat and you feel a sense of accomplishment. Now, you show this to your product manager, stakeholder or any other user who has his/her default browser set to Mozilla Firefox/ Safari/ Opera or any browser other than Google Chrome then you are surprised to notice the UI deviations in website rendering. This implies that your website isn’t as it is working abruptly when accessed through different browsers. The practice to ensure that the layout of a website looks and runs seamlessly across various browsers is called Cross Browser Layout Testing. cross browser compatible We are living in an era where responsive design is turning into a necessity for every website. If you are looking to address the following challenges for Responsive Site Layout in multiple devices for eg., Mobile, Desktop etc., then Galen Framework is one of the best open source framework to choose: Is your text aligned correctly? Is logo aligned properly at top left pane? Is Width and Height of Textbox or any other UI object size is correct? Are you able to play with color Scheme? All GUI show up appropriate in different browsers in different devices In this article, we will take a look on how to set up test automation environment for Galen Framework with IntelliJ and then go ahead with the script. I would also show you how to generate reports for your automation test cases along with few troubleshooting tips to help you in getting started on browser automation with Galen Framework for cross browser layout testing. Let’s dive right in. Step1: Setting up the Environment: The only pre-requisite required would be the installation of JDK 8 in your machine. For installing JDK 8 (minimum JDK version), visit the official website to download by clicking here . After download, installation is simple, just click and continue to finish. Once the installation is complete, you would have to provide a set up the path. Here is how you can do so! Go to windows button and look for “Environment Variables”. Once that is done, you need to select the option to “Edit System Environment Variables”. After that, click on the “New” button for adding “JAVA_HOME” as your installation path. After that, find “Path” in variable section and double-click. In variable value, add “ %JAVA_HOME%\bin” in the path to make sure specified version of java is accessible in the machine. Now, in order to validate if JDK is properly installed or not! Mention the below command in your cmd(Command Prompt): java -version Kudos! You just installed JDK successfully on your machine. Step 2 – Download IntelliJ and Setup Maven Project IntelliJ is a free, open-source Java IDE(Integrated Development Environment) for developing Java-based computer software. You can download Community version of IntelliJ from . here After download, installation is easy, just click next and finish with a default value. In Windows -> Search “IntelliJ”-> A window will open to “Create New Project”. Click on Create New Project, select Maven and click ‘Next’ button. Enter GroupId, ArtifactId and leave version default and click ‘Next’ button. Leave the project name and select the location where your project is to be located. Click on ‘Finish’ button The created project displayed in IntelliJ as below snapshot. Please make sure to enable auto import. Add Maven Dependency in IntelliJ by following the steps Go to Maven repository and search for Galen framework https://mvnrepository.com/ Select the latest Galen Framework Support for Java Test Framework. Select the latest version which lists in the top and copy the Galen Framework dependency which is highlighted in the below snapshot. Copy the above Maven dependency in pom.xml in your project. Once copied, it’s dependencies and transient dependencies are downloaded from the Maven repository. For eg., Dependency like Galen Framework and transient dependencies like Selenium, along with some common libraries. <dependency>
    <groupId>org.testng</groupId>
    <artifactId>testng</artifactId>
    <version> .10</version>
    <scope>test</scope>
</dependency> 6.9 And, the maven project structure looks like below snapshot. Step 3: Implementing a Spec File Understanding of How to write Spec file: Galen spec language is quite flexible and gives you the opportunity to express the exact manner in which you would want your website to behave when accessed through different devices. There are two main parts in the page spec file: Object Definitions and Object Specs. This spec file is important and a pre-requisites to run your automation based on definition added as expected result, which runs and compare actual result in your application. Let me write a simple spec file for the website LambdaTest To create spec file the extension must be “.gspec” The above spec files and its image is copied under specs directory in the resource folder. Note: The spec file will understand if an image is present in the same directory. If not, then you would have to explicitly mention the folder name/image file. However, keep in mind that the sub-folder must reside where your spec files exist. The above spec is just for sample test execution. To more familiarization please visit the site and play with more examples: http://galenframework.com/docs/reference-galen-spec-language-guide/ Step 4: Writing Code We need to write code for our automation script to work with Galen Framework. Before you start writing code for browser automation through Galen framework, you need a directory for your program. You can create a directory by going to Java directory under test. Goto test->java directory-> create a directory name called “testsuites” and create a new class. Here is an example demonstrating browser automation with Galen framework. I will be writing code in order to verify logo present and pixel for LAMBDATEST logo and background colour, font-size for the button that says “START TESTING”. On the IntelliJ window that you are seeing now, write the following code CODE(Self Explanatory): SampleUILayoutTest.java
Last week
Dec , M
Muthuraja Ramakrishnan uploaded an item
Java
SampleUILayoutTest.java testSuites; com.galenframework.api.Galen; com.galenframework.reports.GalenTestInfo; com.galenframework.reports.HtmlReportBuilder; com.galenframework.reports.model.LayoutReport; org.openqa.selenium.WebDriver; org.openqa.selenium.chrome.ChromeDriver; org.testng.annotations.AfterClass; org.testng.annotations.AfterMethod; org.testng.annotations.BeforeClass; org.testng.annotations.Test; java.io.IOException; java.util.Arrays; java.util.LinkedList; java.util.List; { String lambdaTestpage = ; String baseURL= ; WebDriver driver; LayoutReport layoutReport; { System.setProperty( , );
        driver = ChromeDriver(); driver.get(baseURL); driver.manage().window().maximize();
    } { layoutReport = Galen.checkLayout(driver, lambdaTestpage, Arrays.asList( ));
    } { { List<GalenTestInfo> tests = LinkedList<>(); GalenTestInfo test = GalenTestInfo.fromString( ); test.getReport().layout(layoutReport, );
            tests.add(test); HtmlReportBuilder().build(tests, );
        } (Exception e) {
            e.printStackTrace();
        }
    } { driver.quit();
    }

} 27 2018 package import import import import import import import import import import import import import import public class SampleUILayoutTest private static final "src/main/resources/specs/lambdatest.gspec" private static final "https://www.lambdatest.com/" private private /* initialize webdriver, make sure you download chrome driver in http://chromedriver.chromium.org/downloads
    and drop the exe file under resources folder
     */ @BeforeClass public void init () //declaration and instantiation of Objects/variables "webdriver.chrome.driver" "src/main/resources/chromedriver.exe" new //triggers chrome to launch for directing it towards the base url //Maximize the browser window @Test IOException public void checkLogoVisibleAndImageVerification () throws //Executing Layout check and obtaining the Layout report "desktop" @AfterMethod public void reportUpdate () try //Creating a list of tests new //The object you create will be consisting the information regarding the test "Test Automation Using Galen Framework" //Adding layout report to the test report "Verify logo present and log image comparison" //Exporting all test report to html new "target/galen-html-reports" catch @AfterClass public void tearDown () //Close and Quit the driver. Step 5: Report Generation: Once you run the above code, the report gets generated under target ->galen-html-reports. Open reports.html which returns the following report Click on hyperlink label at left side. You will find the detailed report and failures shown on the summary page. Click on any one of the links, for eg., I clicked on the visible link & it validated the below-presented logo. Here is another example, as I clicked on css “background-colour” the background colour was verified as black. Trouble Shooting After setup the project, in case you get any error in IntelliJ, Please double check by navigating to Right-click on Project->Open Module Settings->Modules->Select JDK version 1.8 We should maintain proper indentation in spec file otherwise it may throw an error. Please visit the website for more details. http://galenframework.com/docs/reference-galen-spec-language-guide/ That was all you need to know for using Galen framework to perform cross browser layout testing of your website or web-app. Feel free to share your opinion of this tutorial in the comment section below. getting started with browser automation I would appreciate if you have any questions for me or if you share about your experience with Galen framework for cross browser layout testing has been so far? Happy testing! Previously published at https://www.lambdatest.com/blog/using-galen-framework-for-automated-cross-browser-layout-testing/

Using Galen Framework For Automated Cross Browser Layout Testing

We are living in a bright new age in which sustainability and ethical practises are moving out of the sphere of abstract international treaties such as the Paris agreement, and into the day to day behaviour of consumers and the stores they buy from. . Plastic bags have been banned in various countries across the world , and in France supermarkets face fines for food wastage . However, one persistent outdated practise remains in stores, cafes and gas stations all over the world: the use of paper receipts And it’s not as if we don’t know the damaging effects of continued use. Over the last few years, the issue has been highlighted not once but twice on primetime and has also been featured on leading publications like the , , and , which have all lamented the negative effects that paper receipts have on the shopping experience, the environment, and businesses expenditures. U.S. TV by Jimmy Kimmel BBC The New York Times The Guardian Huffington Post And while some forward thinking leading retailers like Tesco, Apple and are skipping the slips and evolving to digital receipts, the change is still slow. It is time for governments to take steps to ban this practise the same way they have outlawed other harmful practises. Here are three reasons why: CVS Environmental impact From storing digital airline tickets in Apple wallets, to receiving our utility and phone bills via email, the paperless movement is well on its way. And it’s not just because pieces of paper are frustratingly easy to lose, spill coffee on, or damage in our wallet or purse. It is instead because the paper industry is extremely damaging to the environment. While it is easy to argue that such as small piece of paper isn’t going to make much difference in the world, if you look at the effects of the global paper industry, the effects are shocking. According to from , in the United States alone, more than 12.4 million trees and 13 billion gallons of water are used each year in the production of paper receipts, creating 1.5 billion pounds of waste and 4 billion pounds of CO2. research Green America Skip The Slip _Download full report here Why Receipts? P Download full report here Why Receipts? Paper receipts are tiny items with…_www.greenamerica.org Skip the Slip Report: Environmental Costs & Human Health Risks of Paper Receipts with Proposed… But paper receipts are a problem not just because of the waste and destruction they cause, but also for the components that make them. . These chemicals are toxic, and according to recent studies could have harmful effects on humans who touch them. A connects BPA to developmental, reproductive, and neurological problems, which puts workers, consumers, and people involved in collecting trash and recycling at risk. Approximately 93% of paper receipts are coated with BPA — a chemical banned from plastic bottles — or BPS, which means that they should not be recycled study from the New York State Department of Health Consumer impact One of the issues that riles Jimmy Kimmel most is the fact that the excessively long slips are filled with coupons, which consumers almost never use as they are hidden within meters of useless paper. This is a perfect example of the redundancy of paper receipts. They are designed to serve a purpose, whether it be to act as proof of sale, to offer reductions for loyal customers, or to display the contact number of the store. But more often than not, instead of pushing sales and increasing brand loyalty, they are dropped into the trash. The fact is that consumers WANT to use their receipts in the right way. They want to collect coupons and loyalty points. They want to be able to return goods if they break. They want to be able to claim for business expenses when they submit taxes. What they don’t want to do is collect shoe boxes full of paper receipts, or realise one year down the line that the ink has rubbed off the receipt they have so carefully stored, rendering it useless. Furthermore, consumers across all age demographics , and a big part of this comes from taking more care with day to day costs. Assessing weekly expenditures from swathes of folded paper receipts is a time-consuming task which — if we are being honest — most of us forget to do, until the month before tax day when we end up having to pay an accountant to do it for us. are keen to manage their finances better And it’s not as if consumers are resistant to change. According to , 90 percent of customers surveyed would prefer to receive digital receipts when shopping, and half of shoppers would purposefully frequent stores that used digital receipts. And with a range of money management apps available, which can be linked directly with payment systems, life would be so much easier for everyday consumers if paper slips just disappeared. recent studies Commercial impact If damaging the planet and disadvantaging customers weren’t enough of a reason, if anyone is feeling the negative effect of the continued use of receipts, it is stores themselves. By law, in many parts of the world , however doing so with paper is costing stores hundreds if not thousands of dollars each year. This is hitting SMBs particularly hard. stores are required to offer consumers proof of their purchase While the smallest stores are getting hit the hardest, the largest retailers like Tesco, Apple and seem to be making serious moves towards digital receipts. But it’s not just out of the goodness of their hearts. These large retailers know that connecting with consumers digitally, either via sending them receipts online, by giving customers digital loyalty schemes, or unique customer identifiers via opt-in mobile apps, helps them connect better and push deals and discounts to nurture customer relationships and foster brand loyalty. CVS And more than that, a recent Forbes article shows that using apps can also help retailers , allowing them to hit them with the right deals at the right time. understand a individual shopper’s browsing patterns both online and in-store So, with governments around the world taking steps to improve sustainability, reduce waste, support local businesses, make it easier for citizens to comply with tax laws and better manage their finances, there seems to be one small step we could take to make a huge difference to many different aspects of our lives. Let’s join forces with Skip the Slip and Jimmy Kimmel, and pressure retailers to finally put an end to such a wasteful, outdated practise. Sarthak is the founder and CEO of a forward-looking tech firm working on an innovative solution to bring to the retail with its payment terminal. This will allow merchants to easily integrate a digital receipts solution and offer shoppers easily accessible and user-friendly digital receipts without compromising shopper’s privacy or environmental sustainability. TillBilly : tap and go (contactless) digital receipts

Stories, News & Mentions about GREENFRAME

Greenframe’s Stories on HackerNoon