OpenStream

openstream

The steaks we had that night

As you may know, Docker consists of a series of containers that operate independently from each other. It's excellent since all team members can work on the same items. Docker could virtualize OS-level and gather all the needed packages. When we talk about Docker, we are talking about running a program in a container with our custom settings. We could define these settings in a file called Dockerfile. What are the benefits of Docker? Maybe before this, you were using an Apache server, which is good, but it's not suitable and convenient. Docker is good because of: All members are accessing the same packages. Anyone can see the latest changes made by others. All settings and versions are the same. So, with these benefits, we could go to the tutorial. I'm assuming you have Docker installed on your Linux system and are familiar with Laravel. Installing Laravel create a directory and name it what you want. open terminal or CMD in this directory and type in composer create-project laravel/laravel example-app When you installed Laravel successfully, go to the folder of Laravel, and here is your first Laravel structure. go to the public folder and cut all the files there. you must navigate to the root file outside the Laravel folder. paste them here like this: So we did this because we wanted to launch Laravel without the command. From the root, we must establish a primary route to the Laravel folder and have access to all of them. php artisan serve Dockerizing Laravel Now open the index.php file in your root project and alter it like below: So we changed it and placed “Laravel” (the name of our project directory) in all the paths. Simply create a Dockerfile folder at the root (which signifies the root directory of your project) and you're done. It must be precisely this name, with a capital D. Now enter these codes into your Dockerfile: FROM php:7.4-apache COPY ./php.ini /etc/php/7.4/apache2/php.ini RUN a2enmod rewrite WORKDIR /var/www/html RUN docker-php-ext-install mysqli Be aware that you may set the PHP version as you like. Also, the command is used to allow accessing the subfolders in your Apache root system. RUN a2enmod rewrite is the path to your Apache root system. In Windows, this path is whereas, in Linux, it is . WORKDIR C:\HTTPD\Apache24\htdocs /var/www/html In the root, create a php.ini file and leave it empty. It is now time to dockerize our Laravel project. In the root directory where Dockerfile resides, open your terminal and type in . docker build -t laravel:01 is used for creating a new image in Docker. means tag name, which you may set any name for it, and the dot means all your current directories and subdirectories of your project. docker build -t . Now you may verify with the command whether the image was created or not. docker images It's container turn. As you may know, a container is a Docker image for storing and executing multiple packages on Docker. As a consequence, we must design our container for it to operate. So type in this command: docker run -d -p 84:80 -name apachelaravel -v "$PWD":/var/www/html laravel:01 is obvious; it implies constructing and operating the container. indicates setting port 84 to the true port 80 of our Apache system. docker run -p You may verify whether your container is running or not by typing in this command docker container ls -a Now in your browser, simply type in http://localhost:84 If you see an error like this: The stream or file "/var/www/html/laravel/storage/logs/laravel.log" could not be accessed in append mode: unable to open stream: Permission denied The exception occurred when trying to log Don’t worry, you can resolve this simply. Just go to the Laravel directory, open your terminal, and enter these three commands one by one: chmod -R gu+w storage chmod -R guo+w storage php artisan cache:clear These instructions allow access by the Apache server to the storage directory in Laravel. And here you are. Just reload the page, and you will see that Laravel is running successfully.

Dockerize Your Laravel Project in a Jiffy

12 Common Java Mistakes Made by Newcomers

Well, after 2 weeks of waiting I received my Huawei GT-2. Sure, Chinawatch watches you and all, but that was fine with me. My two seconds of research told me it was the superior watch. A 2-week battery life? I had a Samsung watch years ago I returned 2 days later because, after hacking it to interact with every app I had on my phone - including voice replies - the battery life sank to under 2 hours. 2 weeks should translate into.. maybe a day?? I had received a Huawei Watch 2 in a deal with my Huawei P30. Yes, I am Chinese. No, not really. They just know everything about me. Anywho, I created a watch face to track the net yields of my aggregate Coindex trading bots that are live - three of them. I was going to re-learn even more Java to get these values into % average/projected Daily Percentage Returns but wanted to wait for my way-better watch first. Really, no lie. For you code rats out there I had to separate the web request into an async task and load my data there, saving to the object's instances of the  variables. I also had to do some network configuration magic in an .xml. {
    public title = ;
    public btc = ;
    public usd = ;
    private Exception exception;
    public [] onPostExecute() { [] {title, btc, usd};


    }

        private readAll(Reader rd) throws IOException {
            StringBuilder sb = StringBuilder();
            int cp; ((cp = rd.read()) != ) {
                sb.append((char) cp);
            } sb.toString();
        }

        public  JSONObject readJsonFromUrl( url) throws IOException, JSONException {
            InputStream is = URL(url).openStream(); {
                BufferedReader rd = BufferedReader( InputStreamReader(is, Charset.forName( ))); jsonText = readAll(rd);
                JSONObject json = JSONObject(jsonText); json;
            } {
                is.close();
            }
        }


        protected [] doInBackground(Void... voids) { { int responseCode = ;
            JSONObject json = readJsonFromUrl( );
            System.out.println(json.toString());
            JSONArray arr = json.getJSONArray( );
            int count = ;
            Double total = ;
            int countb = ;
            Double totalb = ; (int i = ; i< arr.length(); i++){

                    Double btcstart = (arr.getJSONObject(i).getJSONObject(( )).getDouble( ));
                    Double btca = (arr.getJSONObject(i).getJSONObject(( )).getDouble( ));

                    Double usdstart = (arr.getJSONObject(i).getJSONObject(( )).getDouble( ));
                    Double usda = (arr.getJSONObject(i).getJSONObject(( )).getDouble( ));
                    Double usdp = ((usda / usdstart) - ) * ;
                    Double btcp = ((btca / btcstart) - ) * ;
                    total = total + usdp;
                    count = count + ;
                    countb = countb + ;
                    totalb = totalb + btcp;
            }
            btc = + (totalb / countb);
            usd = + (total / count); [] {title, btc, usd };

        } (Exception e) { RetrieveFeedTask().execute();

            e.printStackTrace();
        } [] { , , };
    }


} < , , []> class RetrieveFeedTask extends AsyncTask Void Void String String "Return %" String "" String "" String return new String String new while -1 return String new try new new "UTF-8" String new return finally String //do stuff try // Day of week 0 "http://jare.cloud:8080/urls" "urls" 0 0.9 0 0.9 for 0 "subobj" "btcstart" "subobj" "btc" "subobj" "usdstart" "subobj" "usd" 1 100 1 100 1 1 "%" "%" return new String catch new return new String "" "" "" And, look how pretty the damn thing is. Leather strap and all. Alright, it's on and updated. Next, we go inside to connect it to PC and enable USB debugging. Right? Right. Wrong! Turns out there is no build number in about to turn on dev options. Uncool, let's google how to do it. Lo and behold folks, there is no USB debugging. There is no Play Store. The damned Chinese made a non-Android watch that syncs with Android devices. Woe is me, folks! It's not real! Alright, now to explain a wee bit better. For my entire adult life I've been assuming everything 90% of the time and being right 75% of the time - leading some CEOs to call me incompetent, this actually saves me a sh#tton of time and I just make up the difference by correcting myself when I'm wrong. Most people have a hard time even admitting they are wrong (ever). I feel this is a redeeming quality of mine. Some don't. In this case, it bit me. I skipped over reviews titled along the lines of 'Huawei GT-2 - Pretty Watch, Not So Smart?' I assumed nobody - not even Huawei spies - would remove Android from Android watches. At this point, I was still wearing my watch out of spite and as a self-loathing exercise. Nay, I wouldn't even return it. I would keep wearing it to punish myself. I realized that as soon as COVID is done I'm shipping to Atlanta, to be an executive and real member of society (on vacation in the USA mind you). I realized that - in high tech and high finance - I couldn't wear this watch. It's embarrassing. Then I realized: F**k 'em. I can recreate the watchface using whatever the heck Huawei uses. And the damn thing has 2wk battery life. Moreover, it displays notifications from whatever damn app I want it to - not just the ones that have bought into Android Wear and made an app. This thing does what I need it to. And better. And longer between annoying recharges. And I can still do my custom ridiculous stuff on it. So screw it. It's an edgelord fashion statement. I don't need Android in me watch.

How I Programmed A Huawei GT-2 Watch To Display Net-Aggregate Crypto-Yields

How to Choose a Stream Processor for Your App based on Advanced Features?

Stream enables you to listen to fee changes in near real-time using SQS, webhooks or websockets. In this tutorial, we will discuss how to use AWS & to respond to feed updates. SQS Lambda While websockets are the preferred method to listen to changes, SQS notifications have a special spot in the feed infrastructure. Stream really shines in its ability to provide real-time because you can interact with the results that you receive. For example, you can listen to for changes to the feed and respond with, for example, an email to a user enticing them to pull the trigger on a purchase, send an SMS message based on an action taken within your application, among hundreds of other scenarios. The goal of this post is for you to walk away with a strong understanding of AWS SQS, Lambda, and of course, Stream. Note: To ensure that you can follow along from start to finish, please create an account or make sure that you have the necessary permissions to use the services mentioned above — we won’t be discussing permissions within AWS. Setting up SQS Setting up AWS SQS is rather straightforward. Once signed in, head on over to the SQS page. From there, we’ll have the option to create a new queue. Click the “Get Started Now” button. Next, you will be presented with a settings page for your new queue. For the name, go ahead and set it to “STREAM”. Then, choose the option on the left (the “Standard Queue). Once created, we’ll need to specify permissions for the newly created STREAM queue. For this tutorial, we’re going to open it up to the world. Note: Never do this — I’m only doing this for example purposes. Always keep your services locked down and only let in applications and users that require access. 😜 Click “Add Permission” and you should be set! Setting up Stream for Use with SQS Before we get started, we’ll need to create a user for Stream. I prefer to create a separate user so that I can change permissions; however, you’re free to use an existing key and secret if you’d like (just skip the first part of this section if that is the case). Create a new IAM User & Role The first step in this process is to head over to the of AWS. Once you are there, click “Users” and then “Add User”. IAM section For the “User name”, enter “STREAM” and for the “Access type”, select “Programmatic access”. Click the next button to continue to the next step. In the above image, you’ll notice that we selected as our Policy name. This will ensure that the user you are creating has access to SQS. You will need to do the same, in addition to setting the “Group name” to “STREAM”. AmazonSQSFullAccess Note: You will need to search for this policy. To do so, simply drop in SQS in the search box. With the “STREAM” role created, continue on to the review stage. Last, go ahead and create a user! Copy the “Access key ID” and “Secret access key” — you’ll need them in the next step. Hook It up to Stream Now, let’s go ahead and drop our API Key and Secret into the Stream dashboard so that Stream backend knows where to route outbound messages. : Steps Go to the Stream dashboard Create a flat feed with the feed name of “user” (be sure to enable notifications) Click on the “user” feed Enable SQS notifications by clicking the button labeled “Active” Fill out the SQS URL, AWS Key, and AWS Secret Click the save button in the top right-hand corner Note: For a full explanation of the Stream SQS feature, you can view the documentation here . Configuring Lambda AWS Lambda lets will allow us to run code without provisioning or managing servers. This service is going to come in handy for decoding the base64 encoded string that is sent to SQS as the payload. Once the message is decoded, you can do anything — send an SMS, email, etc. Let’s kick things off by setting up Lambda. First things first, you’ll want to head over to the Lambda section of AWS (this can easily be found with the search bar). If you’re a new user, go through the short-2 minute tutorial. If you’ve done this before, jump right in. The first step will be to create a new Lambda function as shown in the screenshot below. To copy my configuration, you’ll want to do the following: Select the “Author from scratch” box Name your function “STREAM” Update the Node.js runtime to Node.js v8.10 (outdated, but it is the latest version that is available on Lambda) Specify your “Role” name as “STREAM” Include “SQS Poller” permissions Once you’ve done that, click “Create Function” on the lower right-hand side of the screen. All set! Let’s continue to the next step. If successful, you should see a screen that looks nearly identical to this: To allow the Lambda function to access SQS, there is are some minor configurations that need to be made. Luckily, this step only requires a couple of clicks. Under the “Designer” section, click “SQS” in the scrollable section In the “Configure triggers” section, search for your SQS queue (in our case, it will be named “STREAM”) Keep the batch size at 10 Check “Enable trigger” Finally, click on “Add” Sending SQS Messages to Lambda To connect SQS to Lambda, we’ll use the ARN for the Lambda function. The easiest way to do this is to go to the SQS page, click on “Queue Actions”, and then click on “Trigger a Lambda Function”. Doing so will open a dialog. Select “STREAM” from the dropdown menu and the Lambda Function ARN should automatically populate. Click “Save” once you’re finished. . You can see the status under the “Lambda Triggers” section as shown in the screenshot below. It takes around 30 seconds to 1 minute for the connection to get fully up and running Testing Lambda Functionality 🐑 Now that we have SQS, Lambda and Stream connected, let’s run an end to end test to ensure that messages are making it to the Lambda function. On the Lambda page, specify the contents of your function to be the following: Then click on the “Save” button in the top right-hand corner. Once you’ve successfully saved your Lambda code, you will need to click “Monitoring” which will take you to another page. Once redirected, click “View logs in CloudWatch”. Head back over to the Stream dashboard where you previously configured your SQS settings. There will be a handy button labeled “Test SQS”. Clicking on this will send a test payload to SQS, which then be forwarded over to Lambda where it will be logged out into CloudWatch. Whew! 😅 should What you should see in the payload is an example activity for a tweet activity (example shown above). If nothing showed up in CloudWatch, try reloading. If that doesn’t work, run through the steps outlined above one more time to be sure that you’ve checked every box. Piping the Payload to AWS SNS Let’s do something with the example payload. I’m going to propose that we use AWS SNS to forward the payload as a text message. To enable SNS, we’ll need to add the to the STREAM user. To do that, head back to the IAM section of AWS ( ). From there, click and choose and from the dropdown menu. Once selected, click “Attach Policy” in the bottom right-hand corner. AmazonSNSFullAccess here Groups > STREAM > Add Permissions > Attach Policy AmazonSNSFullAccess AWSLambdaFullAccess Your permissions should now look like this: The STREAM user now has permissions for SNS, so let’s go ahead and set it up. Head over to the and click “Get Started”. Next steps are listed below: SNS section of AWS Steps: Click “Create topic” Set your “Topic name” and “Display name” to STREAM and then click “Create topic” Copy your “Topic ARN” as you’ll need that in a bit Next, we need to modify our Lambda. As you probably noticed, modifying Lambda code in the editor is pretty difficult — or at least it would be on a larger scale. Instead, I recommend creating a new Node.js project somewhere on your computer and installing . This awesome little module has the ability to set up, dry-run, package and deploy your Lambda to AWS without almost no effort. node-lambda Make a directory on your local machine called “stream” and move into that directory. Once inside, run the command “node-lambda setup” and the node-lambda package should bootstrap a Lambda for you. After running setup, it’s a good idea to ignore the generated and files, as well as event.json .env .lambda. Copy and Paste the Following Commands Note: For those who are not familiar with command line, the command above creates a new file (index.js) and installs all of the node modules that we’ll need. Follow the prompts in the terminal to finish up. Modify Your package.json File Open package.json and change scripts section to this: Open .env and Modify Your Keys Inside of the directory you generated, there should be a file and paste your AWS credentials, role ARN (which you’ve copied before) and fill rest of the parameters to match your configuration in AWS. I recommend that you specify the following settings: .env Update Your index.js File Update your index.js file with the following code: Note: We’re still logging out the contents that Stream sends as an example. And most importantly, deploy your code! Fix the SQS Connection When we deployed this time around, it created a Lambda called “STREAM-development” or should have, assuming you followed the directions to a tee. If you did not name your Lambda “STREAM”, follow along with your naming convention. Provided that this is a new Lambda, we’ll need to add a new “trigger”. This is the same as we did previously in the post, so there’s nothing new here. Just click on SQS from the left-hand drawer on the AWS Lambda page (for your new Lambda), and specify the configuration if necessary. Associate Lambda with SNS Topic Remember the ARN from the topic we created? Now is the time to use it. Open your .env file and create a new environment variable called and set the value to your generated SNS Topic ARN. AWS_SNS_TOPIC_ARN Create Your SNS Subscription Before we wrap up and test, we need to generate a subscription for your phone number. This can be done under the SNS section of the AWS dashboard. To create a subscription, click on > and click “Create Subscription. Specify the protocol as SMS and the endpoint should be a valid E.164 formatted phone number (e.g. 1–555–555–5555). topics select your existing topic Upload the Final Code The second to last part of this SNS setup is uploading some updated code to your Lambda function. Simply update your file with the code below and use the command to push it to AWS. index.js yarn deploy All Done! Congratulations! You’ve set up an end to end test using Stream, AWS SQS, Lambda, and SNS to send an SMS message when an event happens to a feed (even if you are the one pressing the “Test SQS” button 😉) I’ve hosted the repo on GitHub, so feel free to clone it if you run into issues: https://github.com/nparsons08/stream-sqs-lambda-sns I’m also available to answer any questions in the comments below. Happy coding! 👋

Using the Stream Real-Time Firehose with AWS SQS, Lambda, and SNS

powers feeds for over 300+ million end users. With all of those users relying on our infrastructure, we’re very good about testing everything that gets pushed into production. Our primary codebase is written in Go, with some remaining bits of Python. Stream Our recent showcase application, , is built with Node.js and we quickly learned that our usual testing methods in Go and Python didn’t quite fit. Furthermore, creating a proper test suite requires a bit of upfront work in Node.js as the frameworks we are using don’t offer any type of built-in test functionality. Winds 2.0 Setting up a good test framework can be tricky regardless of what language you’re using. In this post, we’ll uncover the hard parts of testing with Node.js, the various tooling we decided to utilize in Winds 2.0, and point you in the right direction for when it comes time for you to write your next set of tests. Why Testing is so Important We’ve all pushed a bad commit to production and faced the consequences. It’s not a fun thing to have happen. Writing a solid test suite is not only a good sanity check, but it allows you to completely refactor code and feel confident that your codebase is still functional. This is especially important if you’ve just launched. If you’re working with a team, it’s extremely important that you have test coverage. Without it, it’s nearly impossible for other developers on the team to know if their contributions will result in a breaking change (ouch). Writing tests also encourage you and your teammates to split up code into smaller pieces. This makes it much easier to understand your code, and fix bugs along the way. The productivity gains are even bigger, due to the fact that you catch bugs early on. Finally, without tests, your codebase might as well be a house of cards. There is simply zero certainty that your code is stable. The Hard Parts In my opinion, most of the testing problems we ran into with Winds were specific to Node.js. The ecosystem is always growing. For example, if you are on macOS and run “brew upgrade” (with homebrew installed), your chances of seeing a new version of Node.js are quite high. With Node.js moving quickly and libraries following close behind, keeping up to date with the latest libraries is difficult. Below are a few pain points that immediately come to mind: Testing in Node.js is very opinionated and un-opinionated at the same time. Many people have different views on how a test infrastructure should be built and measured for success. The sad part is that there is no golden standard (yet) for how you should approach testing. There are a large number of frameworks available to use in your application. However, they are generally minimal with no well-defined configuration or boot process. This leads to side effects that are very common, and yet hard to diagnose; so, you’ll likely end up writing your own test runner from scratch. It’s almost guaranteed that you will be to write your own test runner (we’ll get to this in a minute). required The situations listed above are not ideal and it’s something that the Node.js community needs to address sooner rather than later. If other languages have figured it out, I think it’s time for Node.js, a widely adopted language, to figure it out as well. Writing Your Own Test Runner So… you’re probably wondering what a test runner . To be honest, it’s not that complicated. A test runner is the highest component in the test suite. It allows for you to specify global configurations and environments, as well as import fixtures. One would assume this would be simple and easy to do… Right? Not so fast… is What we learned is that, although there is a solid number of test frameworks out there, not a single one for Node.js provides a unified way to construct your test runner. Sadly, it’s up to the developer to do so. Here’s a quick breakdown of the requirements for a test runner: Ability to load different configurations (e.g. local, test, development) and ensure that you load a production configuration — you can guess what goes wrong when that happens. NEVER Lift and seed a database with dummy data for testing. This must work for various databases, whether it be MySQL, PostgreSQL, MongoDB, or any other, for that matter. Ability to load fixtures (files with seed data for testing in a development environment). With Winds, we chose to use Mocha as our test runner. Mocha provides an easy and programmatic way to run tests on an ES6 codebase via command-line tools (integrated with Babel). To kick off the tests, we register the Babel module loader ourselves. This provides us with finer grain greater control over which modules are imported before Babel overrides Node.js module loading process, giving us the opportunity to mock modules before any tests are run. Additionally, we also use Mocha’s test runner feature to pre-assign HTTP handlers to specific requests. We do this because the normal initialization code is not run during tests (server interactions are mocked by the Chai HTTP plugin) and run some safety check to ensure we are not connecting to production databases. While this isn’t part of the test runner, having a fixture loader is an important part of our test suite. We examined existing solutions; however, we settled on writing our own helper so that it was tailored to our requirements. With our solution, we can load fixtures with complex data-dependencies by following an easy ad-hoc convention when generating or writing fixtures by hand. Tooling for Winds Although the process was cumbersome, we were able to find the right balance of tools and frameworks to make proper testing become a reality for our backend API. Here’s what we chose to go with: Mocha ☕ , described as a “feature-rich JavaScript test framework running on Node.js”, was our immediate choice of tooling for the job. With well over 15k stars, many backers, sponsors, and contributors, we knew it was the right framework for the job. Mocha Chai 🥃 Next up was our assertion library. We chose to go with the traditional approach, which is what works best with Mocha — . Chai is a BDD and TDD assertion library for Node.js. With a simple API, Chai was easy to integrate into our application and allowed for us to easily assert what we should to be returned from the Winds API. Best of all, writing tests feel natural with Chai. Here’s a short example: Chai expect Sinon 🧙‍ With the ability to work with any unit testing framework, was our first choice for a mocking library. Again, a super clean integration with minimal setup, Sinon turns mocking requests into a simple and easy process. Their website has an extremely friendly user experience and offers up easy steps to integrate Sinon with your test suite. Sinon Nock 🔮 For all external HTTP requests, we use , a robust HTTP mocking library that really comes in handy when you have to communicate with a third party API (such as ). There’s not much to say about this little library aside from the fact that it is awesome at what it does, and that’s why we like it. Here’s a quick example of us calling our engine for Stream: nock Stream’s REST API personalization Mock-require 🎩 The library allows dependencies on external code. In a single line of code, you can replace a module and mock-require will step in when some code attempts to import that module. It’s a small and minimalistic, but robust library, and we’re big fans. mock-require Istanbul 🔭 is a JavaScript code coverage tool that computes statement, line, function and branch coverage with module loader hooks to transparently add coverage when running tests. Although we have similar functionality with CodeCov (see next section), this is a nice tool to have when running tests locally. Istanbul The End Result — Working Tests With all of the libraries, including the test runner mentioned above, let’s have a look at what a full test looks like (you can have a look at our entire test suite here ): Continuous Integration There are a lot of continuous integration services available, but we like to use because they love the open-source environment just as much as we do. Given that Winds is open-source, it made for a perfect fit. Travis CI Our integration is rather simple — we have a file that sets up the environment and kicks off our tests via a simple command. The coverage reports back to GitHub, where we have a clear picture of whether or not our latest codebase or PR passes our tests. The GitHub integration is great, as it is visible without us having to go to Travis CI to look at the results. Below is a screenshot of GitHub when viewing the PR (after tests): .travis.yml npm In addition to Travis CI, we use a tool called . CodeCov is similar to , however, it’s a visualization tool that allows us to easily see code coverage, files changed, lines modified, and all sorts of other goodies. Though visualizing this data is possible without CodeCov, it’s nice to have everything in one spot. CodeCov Istanbul What We Learned We learned a lot throughout the process of developing our test suite. With no “correct” way of doing things, we decided to set out and create our own test flow by sorting through the available libraries to find ones that were promising enough to add to our toolbox. What we ultimately learned is that testing in Node.js is not as easy as it may sound. Hopefully, as Node.js continues to grow, the community will come together and build a rock solid library that handles everything test related in a “correct” manner. Until then, we’ll continue to use our test suite, which is open-source on the . Winds GitHub repository Limitations No Easy Way to Create Fixtures Frameworks and languages, such as Python’s Django, have easy ways to create fixtures. With Django, for example, you can use the following commands to automate the creation of fixtures by dumping data into a file: ./manage.py dumpdata > db.json The Following command will dump the whole database into a db.json file: :./manage.py dumpdata admin.logentry > logentry.json The Following command will dump only the content in django admin.logentry table **The Following command will dump the content in django auth.user table:**./manage.py dumpdata auth.user > user.json There’s no easy way to create a fixture in Node.js. What we ended up doing is using MongoDB Compass and exporting JSON from there. This resulted in a nice fixture, as shown below (however, it was a tedious process and prone to error): Unintuitive Module Loading When Using Babel, Mocked Modules, and Mocha Test-Runner To support a broader variety of node versions and have access to latest additions to Javascript standard, we are using Babel to transpile our ES6 codebase to ES5. Node.js module system is based on the CommonJS standard whereas the ES6 module system has different semantics. Babel emulates ES6 module semantics on top of the Node.js module system, but because we are interfering with module loading by using mock-require, we are embarking on a journey through weird module loading corner cases, which seem unintuitive and can lead to multiple independent versions of the module imported and initialized and used throughout the codebase. This complicates mocking and global state management during testing. Inability to Mock Functions Used Within the Module They Are Declared in When Using ES6 Modules When a module exports multiple functions where one calls the other, it’s impossible to mock the function being used inside the module. The reason is that when you require an ES6 module you are presented with a separate set of references from the one used inside the module. Any attempt to rebind the references to point to new values does not really affect the code inside the module, which will continue to use the original function. Final Thoughts Testing Node.js applications is a complicated process because the ecosystem is always evolving. It’s important to stay on top of the latest and greatest tools so you don’t fall behind. There are so many outlets for JavaScript related news these days that it’s hard to keep up to date with all of them. Following email newsletters such as and is a good start. Beyond that, joining a subreddit such as is a great idea. If you like to stay on top of the latest trends, does a great job at helping developers visualize trends in the testing world. JavaScript Weekly Node Weekly /r/node State of JS Lastly, here are a couple of my favorite blogs where articles often popup: Hacker Noon Free Code Camp Bits and Pieces Think I missed something important? Let me know in the comments, or on Twitter – . @NickParsons Also, if you’d like to check out Stream, we have a great 5 minute tutorial on our website. Give it a shot . here

Testing Node.js in 2018

is an API that enables developers to build news feeds and activity streams ( ). We are used by over 500 companies and power the feeds of more than 300 million end users. Companies such as Product Hunt, , Powerschool, , , Compass and Fabric (Google) rely on Stream to power their news feeds. In addition to the API, the founders of Stream also wrote the most widely used . Stream try the API Under Armour Bandsintown Dubsmash open source solution for building scalable feeds Here’s what Stream looks like today: Number of servers: 180 Feed updates per month: 34 billion Average API response time: 12ms Average real-time response time: 2ms Regions: 4 (US-East, EU-West, Tokyo and Singapore) 1B API requests per month (~20k/minute) Given that most of our customers are engineers, we often talk about our stack. Here’s a high level overview: is our primary programming language. Go We use a custom solution built on top of + for our primary database (we started out on but wanted more control over performance). stores configs, API keys, etc. RocksDB Raft Cassandra PostgreSQL and handle tracing, and provide detailed monitoring and we use the stack for centralized logging. OpenTracing Jaeger StatsD Grafana ELK is our language of choice for machine learning, devops and our website ( , & ). Python https://getstream.io Django DRF React Stream uses a combination of and . This results in fast read performance when a users open their feed, as well as fast propagation times when a famous user posts an update. fanout-on-write fanout-on-read and I ( ) are the developers who started Stream nearly 3 years ago. We founded the company in Amsterdam, participated in Techstars NYC 2015 and opened up our Boulder, Colorado office in 2016. It’s been quite a crazy ride in a fairly short amount of time! With over 15 developers and a host of supporting roles, including sales and marketing, the team feels enormous compared to the early days. Tommaso Barguli @tschellenbach The Challenge: News Feeds & Activity Streams The nature of follow relationships makes it hard to scale feeds. Most of you will remember Facebook’s long load times, Twitter’s fail whale or Tumblr’s year of technical debt. Feeds are hard to scale since there is no clear way to shard the data. Follow relationships connect everyone to everyone else. This makes it difficult to split data across multiple machines. If you want to learn more about this problem, check out these papers: Twitter’s tech back in the days with 150 million active users LinkedIn’s ranked feeds Yahoo/Princeton research paper At a very high level there are 3 different ways to scale your feeds: Fanout-on-write: basically precompute everything. It’s expensive, but its easy to shard the data. Fanout-on-read: hard to scale, but more affordable and new activities show up faster. Combination of the above two approaches: better performance and reduced latency, but increased code complexity. Stream uses a combination of fanout-on-write and fanout-on-read. This allows us to effectively support both customers with highly connected graphs, as well as customers with a more sparse dataset. This is important since the ways in which our customers use Stream are very different. Have a look at these screenshots from , Unsplash, and : Bandsintown Product Hunt Switching from to Python Go After years of optimizing our existing feed technology we decided to make a larger leap with 2.0 of Stream. While the first iteration of Stream was powered by Python and Cassandra, for Stream 2.0 of our infrastructure we . The main reason why we switched from Python to Go is performance. Certain features of Stream such as aggregation, ranking and serialization were very difficult to speed up using Python. switched to Go We’ve been using Go since March 2017 and it’s been a great experience so far. Go has greatly increased the productivity of our development team. Not only has it improved the speed at which we develop, it’s also 30x faster for many components of Stream. The performance of Go greatly influenced our architecture in a positive way. With Python we often found ourselves delegating logic to the database layer purely for performance reasons. The high performance of Go gave us more flexibility in terms of architecture. This led to a huge simplification of our infrastructure and a dramatic improvement of latency. For instance, we saw a 10 to 1 reduction in web-server count thanks to the lower memory and CPU usage for the same number of requests. Initially we struggled a bit with package management for Go. However, using together with the contributed to creating a great workflow. Dep VG package If you’ve never tried Go, you’ll want to try this online tour: https://tour.golang.org/welcome/1 Go as a language is heavily focused on performance. The built-in tool is amazing for finding performance issues. Uber’s library is great for visualizing data from PPROF and will be bundled in PPROF in Go 1.10. PPROF Go-Torch Switching from Cassandra to & Raft RocksDB 1.0 of Stream leveraged for storing the feed. Cassandra is a common choice for building feeds. Instagram, for instance started, out with but eventually to handle their rapid usage growth. Cassandra can handle write heavy workloads very efficiently. Cassandra Redis switched to Cassandra Cassandra is a great tool that allows you to scale write capacity simply by adding more nodes, though it is also very complex. This complexity made it hard to diagnose performance fluctuations. Even though we had years of experience with running Cassandra, it still felt like a bit of a black box. When building Stream 2.0 we decided to go for a different approach and build . Keevo is our in-house key-value store built upon RocksDB, gRPC and Raft. Keevo is a highly performant embeddable database library developed and maintained by Facebook’s data engineering team. RocksDB started as a fork of Google’s LevelDB that introduced several performance improvements for SSD. Nowadays RocksDB is a project on its own and is under active development. It is written in C++ and it’s fast. Have a look at how this . In terms of technology it’s much more simple than Cassandra. This translates into reduced maintenance overhead, improved performance and, most importantly, more consistent performance. It’s interesting to note that also uses RocksDB for . RocksDB benchmark handles 7 million QPS LinkedIn their feed Our infrastructure is hosted on and is designed to survive entire availability zone outages. Unlike Cassandra, Keevo clusters organizes nodes into leaders and followers . When a leader (master) node becomes unavailable the other nodes in the same deployment will start an election and pick a new leader. Electing a new leader is a fast operation and barely impacts live traffic. AWS To do this, Keevo implements the Raft consensus algorithm using . This ensures that every bit stored in Keevo is stored on 3 different servers and operations are always consistent. This site does a great job of visualizing how Raft works: Hashicorp’s Go implementation https://raft.github.io/ Not Quite Microservices By leveraging Go and RocksDB we’re able to achieve great feed performance. The average response time is around 12ms. The architecture lies somewhere between a monolith and a microservice. Stream runs on the following 7 services: Stream API Keevo Real time & Firehose Analytics Personalization & Machine learning Site & Dashboard Async Workers To see all our services divided into stacks, head over . here Personalization & Machine Learning Almost all large apps with feeds use machine learning and personalization. For instance, LinkedIn prioritizes the items in your feed. Instagram’s explore feed displays pictures outside of the people you follow that you might be interested in. Etsy uses a similar approach to optimize ecommerce conversion. Stream supports the following 5 use cases for personalization: Documentation for building personalized feeds. All of these personalization use cases rely on combining feeds with analytics and machine learning. For the machine learning side we generate the models using . The models are different for each of our enterprise customers. Typically we’ll use one of these amazing libraries: Python — LightFM How to implement recommender systems XGBoost Scikit learn , and Dask Numpy Pandas (For development) Jupyter Notebook Mesa Framework Analytics Analytics data is collected using a tiny Go-based server. In the background, it will spawn go-routines to rollup the data as needed. The resulting metrics are stored in . In the past, we looked at , which seems like a solid project. For now, we could get away with a simpler solution though. Elastic Druid Dashboard & Site The dashboard is powered by and . We also use React and Redux for all of our example applications: React Redux Cabin: A tutorial series on building a social network with React & Redux (Winds 2.0 is coming out very soon) Winds: Open source RSS & Podcast app The site, as well as the API for the site, is powered by , and . Stream is sponsoring Django Rest Framework since it’s a pretty great open source project. If you need to build an API quickly there is no better tool than DRF and Python. Python Django Django Rest Framework We use to resize the images on our site. For us, Imgix is cost efficient, fast and overall a great service. is a good open source alternative. Imgix Thumbor Real time Our real time infrastructure is based on Go, Redis and the excellent library. It implements the Bayeux protocol. In terms of architecture it’s very similar to the node based library. gorilla websocket Faye It was interesting to read the “ ” post on Hacker News. The author moves from Go to Node to improve performance. We actually did the exact opposite and moved from Node to Go for our real time system. The new Go-based infrastructure handles 8x the traffic per node. Ditching Go for Node.js Devops, Testing & Multiple Regions In terms of devops the provisioning and configuration of instances is fully automated using a combination of: CloudFormation Cloud-Init Puppet Boto & Fabric Because our infrastructure is defined in code it has become trivial to launch new regions. We heavily use . Every single piece of our stack is defined in a CloudFormation template. If needed we are able to spawn a new dedicated shard in a few minutes. In addition, AWS Parameter Store is used to hold application settings. Our largest deployment is in US-East, but we also have regions in Tokyo, Singapore and Dublin. CloudFormation A combination of Puppet and Cloud-init is used to configure our instances. We run our self-contained Go’s binaries directly on the EC2 instance without any additional containerization layer. Releasing new versions of our services is done by . Travis first runs our test suite. Once it passes, it publishes a new release binary to . Common tasks such as installing dependencies for the Go project, or building a binary are automated using plain old Makefiles. (We know, crazy old school, right?) Our binaries are compressed using UPX. Travis GitHub Tool highlight: Travis has come a long way over the past years. I used to prefer in some cases since it was easier to debug broken builds. With the addition of the aptly named “debug build” button, Travis is now the clear winner. It’s easy to use and free for open source, with no need to maintain anything. Travis Jenkins Next we use to do a rolling deploy to our AWS instances. If anything goes wrong during the deploy it will halt the deploy. We take stability very seriously: Fabric A high level of test coverage is required. Releases are created by Travis (making it hard to deploy without running tests). Code is reviewed by at least 2 team members. Our extensive QA integration test suite evaluates if all 7 components still work. We’ve written about our experience with . When things do break we do our best to be transparent about the ongoing issue: testing our Go codebase StatusPage for 24/7 phone support Twilio for notifying our team VictorOps #firefighting channel on Slack for customer support Intercom VictorOps is a recent addition to our support stack. It’s made it very easy to collaborate on ongoing issues. Tool Highlight: VictorOps The best part about is how they use a timeline to collaborate amongst team members. VictorOps is an elegant way to keep our team in the loop about outages. It also integrates well with . This setup enables us to quickly react to any problems that make it into production, work together and resolve them faster. VictorOps Slack The vast majority of our infrastructure runs on AWS: for load balancing ELB for reliable Postgres hosting RDS for Redis hosting ElastiCache for our DNS Route53 for our CDN Cloudfront for ELK and tracing with AWS ElasticSearch Jaeger The devops responsibilities are shared across our team. While we do have one dedicated devops engineer, all our developers have to understand and own the entire workflow. Monitoring Stream uses for tracing and for beautiful dashboards. The tracking for Grafana is done using . The end result is this beauty: OpenTracing Grafana StatsD We track our errors in and use the ELK stack to centralize our logs. Sentry Tool Highlight: OpenTracing + Jaegar One new addition to the stack is . In the past we used , which works like a charm for Python, but isn’t able to automatically measure tracing information for Go. OpenTracing with is a great solution that works very well for Stream. It also has, perhaps, the best logo for a tracing solution: OpenTracing New Relic Jaeger Closing Thoughts is an absolutely amazing language and has been a major win in terms of performance and developer productivity. For tracing we use OpenTracing and Jaeger. Our monitoring is running on StatsD and Graphite. Centralized logging is handled by the ELK stack. Go Stream’s main database is a custom solution built on top of and Raft. In the past we used Cassandra, which we found hard to maintain and which didn’t give us enough control over performance when compared to RocksDB. RocksDB We leverage external tools and solutions for everything that’s not a core competence. hosting is handled by ElastiCache, Postgres by RDS, email by , test builds by Travis and error reporting by Sentry. Redis Mailgun Thank you for reading about our stack! If you’re a user of Stream, please be sure to add to your stack on StackShare. If you’re a talented individual, ! And finally, if you haven’t tried out Stream yet, take a look at this quick . Stream come work with us tutorial for the API This post was originally written by Thierry Schellenbach, CEO at GetStream.io . The original post can be found at https://stackshare.io/stream/stream-and-go-news-feeds-for-over-300-million-end-users .

Stream & Go: News Feeds for Over 300 Million End Users

Brought to you by Stream . XKCD making fun of Facebook’s original Move Fast and Break Things motto Disclosure: , the API for scalable feeds, has previously sponsored Hacker Noon. Stream is used in production by more than 500 companies and 200 million end users. While we like to move fast, we definitely don’t like to break things. Stream’s API An extensive test infrastructure enables us to move quickly and deploy code with confidence. A solid testing workflow is essential to stay productive as your team and grows. codebase Most of the services powering Stream are written in Go. This blogpost will cover in detail what we learned writing tests for a large Go codebase. Our testing workflow Testing is a core part of our development process. Not a single line of code is deployed to a live system before it’s properly tested and peer reviewed. Our workflow looks like this: Implement the new feature with TDD using Go’s package, alongside some goodies from testing stretchr/testify Write integration tests and acceptance tests with and our own soon-to-be-released library onsi/ginkgo bdd Wait for the green light from Travis CI Check test coverage on Codecov Ship the new feature to staging Unit testing While following the TDD approach when writing our code, we make use of Go’s standard library package to write all of our tests: its simplicity matches idiomatic Go code, and more importantly, it works perfectly with table tests, which we use extensively. testing As experienced programmers already know, table tests are a simple way to perform multiple input/output tests against a method or behavior with a small code fingerprint. Also, thanks to Go’s anonymous and inline structs, the code looks simple and easy to extend: testCases := []struct{  input          string  expectedOutput int}{  {    input:          "aabbcc",    expectedOutput: 3,  },  {    input:          "abcdefg",    expectedOutput: 1,  },  ...}for _, tc := range testCases {  output := getRepetitions(tc.input)  assert.Equal(t, tc.expectedOutput, output)} Most Gophers will be familiar with Go’s pattern for handling errors, which is used also when writing tests: call a method, grab the error, check whether it’s not , and act consequently (happy paths to the left!). nil Writing tests with the package goes the same way, but we decided to add something on top of it. testing Let’s say we’re testing a function that checks if the headers of an HTTP request are well-formed, and we’re working with a table-based test: IsValid(*http.Request) (bool, error) func TestIsValid(t *testing.T) {        ...        for _, tc := range testCases {                t.Run(tc.name, func(t *testing.T) {                        valid, err := IsValid(tc.req)                        if tc.shouldError && err == nil {                                t.Fatal("expected error, got none")                        }                        if !tc.shouldError && err != nil {                                t.Fatalf("expected no error, got %s", err)                        }                        if valid != tc.expected {                                t.Errorf("expected valid to be %t, got %t", tc.expected, valid)                        }                })        }} We found that the usual Go idiomatic error checking adds just too much noise in tests, and this is why we adopted ‘s and packages all across our unit tests. testify assert require The previous snippet becomes: import (        "github.com/stretchr/testify/assert"        "github.com/stretchr/testify/require")func TestIsValid(t *testing.T) {        ...        for _, tc := range testCases {                t.Run(tc.name, func(t *testing.T) {                        valid, err := IsValid(tc.req)                        if tc.shouldError {                                require.Error(t, err)                        } else {                                require.NoError(t, err)                        }                        assert.Equal(t, tc.expected, valid)                })        }} The test is now cleaner and easier to extend. As a general rule we rely on when testing preconditions (i.e.: checking the error returned by a function before continuing) and for actual logic assertions. This ensures that our tests always return meaningful results, and we don’t have to waste time debugging our own tests. require assert In some places across our codebase we decided to go further and to use the full package. This helps us in writing some peculiar tests faster. testify/suite Pro Tip: When working with table tests, include a field in your table containing a brief description of the entry that’s being tested: using it as the parameter of the method will make you and your teammates able to understand tests in a glance, both when looking at the code as well as when looking at the command output. name name testing.T.Run test func TestTable(t *testing.T) {        testCases := []struct {                name        string                dividend    float64                divisor     float64                shouldError bool        }{                {                        name:        "dividing by zero",                        shouldError: true,                        dividend:    42,                        divisor:     0,                },                {                        name:        "dividing by odd numbers",                        shouldError: false,                        dividend:    42,                        divisor:     21,                },                ...        }        for _, tc := range testCases {                t.Run(tc.name, func(t *testing.T) {                        ...                })        }} Mocking Writing mocks and “sandboxed” behaviors is a vital part of unit testing, which becomes increasingly important as the complexity of your code grows. We decided to use two approaches to mocking, depending on the “cost” of the process: direct dependency injection or the GoMock framework . Dependency injection is a common software design pattern in Go, and it comes handy when dealing with mocking. By using interfaces, Go’s arguably most powerful feature, stubbing behaviors and mocking components becomes quite straightforward. Dependency injection Let’s say we have this component which performs caching on : DB Redis type DB struct {        cache *redis.Client}func (d *DB) Get(query string) (string, error) {        ...        cached, err := d.cache.Get(query).Result()        if err == redis.Nil {                return cached, nil        }        ...} When testing the actual behavior of the method at some point we probably won’t need an actual Redis connection: we need then to mock the behavior of the field in our struct. We can easily rearrange the previous code so to use an interface we can then re-implement: DB.Get cache DB type DB struct {        cache Cache}type Cache interface {        Get(string) (string, error)}type RedisCache struct {        client *redis.Client}func (c *RedisCache) Get(key string) (string, error) {        v, err := c.client.Get(query).Result()        if err == redis.Nil {                return v, nil        }        return v, err} We also consequently rearrange the affected parts: DB.Get func (d *DB) Get(query string) (string, error) {        cached, err := d.cache.Get(query)        if err != nil {                return "", err        }        if cached != "" {                return cached, nil        }        ...} We can now mock the cache in our tests by just implementing the interface with a mock structure which behaves as we need: Cache type mockCache struct{}func (m mockCache) Get(string) (string, error) {        return "", nil}func TestDBGet(t *testing.T) {        mock := mockCache{}        db := &DB{cache: mock}        db.Get(...)        ...} golang/mock There are some scenarios where the previous approach would be too complicated and/or steal too much time (or, even worse, obfuscate the underlying code). In such cases, we create “proper” mocks with the framework: with a single command, , it generates full type mocks which we can use in our tests by settings expectations about received calls and outputs. GoMock mockgen Our typical use case is when dealing with : refactoring our code to have simple, interface-based components would add too much noise, and with GoMock we basically get fully functional client and server mocks basically for free, which perfectly match the kind of tests you want to perform on an input-output system like an RPC framework. gRPCs GoMock is a powerful tool that simplifies Gophers’ lives a lot, and we encourage you to play around with it and to take a look at the . examples on the project’s Github repository Integration tests We don’t perform integration tests nor acceptance tests with Go’s default library. We instead separate them from the rest of the codebase and place them in a separate repository which has its own lifecycle and dedicated CI jobs. Doing so allows us to perform really careful feature tests, performance tuning, integrating with the other moving parts involved, quality assurance tasks, and general end-to-end tests and benchmarks. At first, we decided to use for these kinds of tests: Ginkgo is a great open source contribution that allows BDD for Go in a quite handy way. Go’s peculiarities make it somewhat more complicated than other languages (like Python and Ruby) to write behavior-driven code, but in the end we found it was a good trade between ease of use and effectiveness. Ginkgo We tuned our setup and came up with the following structure: every feature/component that we want to test lives in a dedicated package inside a folder, and we can easily run specific tests with some Makefile magic. This includes many options for targeting specific tests with regular expressions (using the and Ginkgo flags), triggering parallel execution, detecting races and so on. features/ --focus --skip But after some time we concluded that raw Ginkgo has some shortcomings for our purposes. Ultimately we wanted to write better “specs”, but Ginkgo tests had the following issues for us: Get noisy pretty quickly Don’t incentivize strong opinions when it comes to DSL and BDD best practices Don’t have a (default) informative output (but it can be extended with a custom ) Reporter Have output which is not really suitable for documentation Don’t have built-in reporting Have an “unfriendly” approach to fine-grained tests So, we started experimenting and came up with a homemade library, partially built on top of Ginkgo and . We call it simply , and we use it everywhere. It’s going to be open sourced and publicly released as soon as possible, but it’s not there yet; consider this a sneak peak! Gomega matchers bdd has two goals: writing specs a-là RSpec and writing feature tests a-là Cucumber, with all the benefits of these two famous tools. It exposes features we like about Ginkgo and hides the ones we don’t like, also making some strong assumptions about how we want to write our “specs”: bdd A function is the container of a whole set of logically-connected tests (like the ones related to a single broad feature). Spec A contains any number of blocks, defining “things to be tested”. Spec Describe A block contains any number of blocks (or other blocks). Describe When Describe A block contains a scenario to be tested for the current “thing”. It can also contain sub-scenarios as blocks. When Describe All assertions are done inside a function. Should Writing tests with this structure is really important for a correct “sentence” formulation of these tests, and allows us to write our tests faster while better understanding what’s going on. as a company is growing fast, which means onboarding new hires and adding more tests to our codebase. We are new teammates to feel comfortable writing tests and not having to waste time wondering how to write them consistently (especially during their first days at Stream). Our humble testing library is written with them in mind! Stream We now have a consistent, no-nonsense “testing grammar” that is familiar to every developer in the company. Having a library that ensures your tests are always aligned with your teammates’ ones results in better code reviews, easier report checking, and overall faster development cycles. When writing specs, it all starts with a function: Spec var _ = Spec("Adding activities", func() {        When("adding to a feed group that is not configured", func() {                var err error                BeforeEach(func() { feedGroup = "user_bogus_group" })                JustBeforeEach(func() {                        _, err = feed.AddActivity(&gestream.Activity{                                Actor:  "john",                                Verb:   "like",                                Object: "apples",                        })                })                Should("return a clear error", func() {                        Expect(err).To(HaveOccurred())                        Expect(err).To(BeFeedConfigExceptionError())                        Expect(err).To(BeAnErrorWithDetailMsg("user_bogus_group feed group does not exists"))                })        })}) As said before, we ensure that a block always contains any number of blocks, and that a always contains some blocks. Describe When When Should We also make sure that the string descriptions for every block don’t stutter with the verb they’re referring to, so to always get a nice output: we think that reading the output of the tests is really important down the road, so the syntax of the single expectations should always be formulated and read in proper English. We want our tests to be familiar to read even months after writing them: you really don’t want to read awful things like , but rather «when if I am logged in I has there is a logout button» « . when I am logged in, the logout button should be visible» Speaking of the output, we wrote a custom Ginkgo reported inspired by documentation format, and it looks something like this in the console: RSpec FlatFeeds Suite  Flat Feeds    ...    adding activities:      when verb field is missing        should return an explicit error      when verb field is too long        should return an explicit error      when no pagination is specified        should return 25 activities by default        when using the id_lt pagination parameter          should have id_lt equal to the second activity from the top        when using the id_gt pagination parameter          should have id_gt equal to the second activity from the top    ...Finished in 3.8849 seconds142 examples, 0 failures, 3 skippedPASS And this is combined with HTML reporting and integration with our CI, so that we always know what’s happening, when, where, and (rarely 😏) why it’s failing. On top of this, we’re busy working on the second purpose of , which is Cucumber-like feature tests. This part is completely homemade and doesn’t use Ginkgo (although it does use Gomega matchers), and allows us to write feature files directly in Go, while being as close as possible as the neat feature files you’re probably familiar with. bdd Feature("sample feature", func() {        Scenario("login",                        Given("i am not logged in", givenNotLoggedIn()).                                And("i am on the home page", givenOnHomePage()).                                And("i am using a mobile browser", givenOnMobileBrowser()),                        When("entering my email and password", enterEmailAndPassword()),                        Then("i should be logged in",                                func() {                                        Expect(me).To(BeLoggedIn())                                }),                )}) This is still in active development: you’ll hear more in the upcoming weeks… Wrapping up We believe that only great tests can lead to great software: writing tests is a compelling task and there are no universal rules about it. Our workflow is a constant work-in-progress refinement and tweaking process, because as we write more code, we discover new ways of testing it. Writing tests with Go is generally a fun experience, thanks to the simplicity of the language: the standard library offers a streamlined testing environment, and the vibrant Gophers community came up with nice tools built on top of it. Nevertheless, Go is a young language and the when it comes to advanced testing techniques it’s far from offering the commodities you find in other languages like Python or Ruby: there’s a lot of room for experimenting and trying out new concepts, and that’s why we’re investing a lot of efforts in developing our own testing gear. So now you know how we deal with testing with Go, and we hope you’re inspired for writing better code and maybe adjusting your programming routines. Good luck!

Go testing at Stream

an Object and got its class name @ its hexadecimal hashcode? Have you ever System-d dot out Chances are: it wasn’t what you were hoping for and it didn’t you help much. The problem is that, in Java, we lack a function which generates a good and consistent textual representation for any Java object. If we had it, we would write: System.out.println( .apply(myObject) ); toString instead of System.out.println( myObject**.toString()** ); // finger crossed, we// have a good toString. We would then stop cursing at developers for not implementing a damn good . toString is easy, if you decide to ignore a few edge cases. Hacking a universal toString It just requires you to use the reflection API to iterate over all the object’s fields to print them. that you can use with , with and in any Java 8 program. I wrote one Nashorn jjs nudge4j It’s written as a Nashorn JavaScript function. The function pretty much delegates all of the work to the poorly named (part of ). ReflectionToStringBuilder Apache Commons Lang You don’t need Apache Commons Lang in your classpath (I don’t have it either). I hate dependencies and I am not going to ask you to add it just for this. The code comes with to load it on demand from the maven repo. a good trick How do we use the toString function ? In Nashorn jjs and nudge4j you load it and you are ready to go: load(" ")toString(java.lang.Thread.currentThread()); https://gist.githubusercontent.com/lorenzoongithub/d1a7e047b29aef79d363b3b56c2e666a/raw/c486452e25d0f419101ae7bddd259240b17f1762/toString.js In Java you need to write a bit of boiler plate to wrap it up as a Java Function: Function<Object, String> toString = new Function<Object,String>() {private ScriptEngine engine; {try {engine = new ScriptEngineManager().getEngineByName("JavaScript");try (InputStream is = new URL(" ").openStream()) { https://gist.githubusercontent.com/lorenzoongithub/d1a7e047b29aef79d363b3b56c2e666a/raw/c486452e25d0f419101ae7bddd259240b17f1762/toString.js engine.eval(new InputStreamReader(  
      is, StandardCharsets.UTF\_8));  
  }  
} catch (ScriptException | IOException e) {  
  throw new RuntimeException(e);  
} } public String apply(Object oj) {try {return ""+((Invocable)engine).invokeFunction("toString", oj);} catch (NoSuchMethodException | ScriptException e) {throw new RuntimeException(e);}}}; System.out.println(toString.apply(Thread.currentThread())); : a universal function to stringify any Java Object. And that’s it It won’t have an impact on your day to day work but it might turn handy on one of those days. debugging Debug less. loading toString.js in nudge4j

Stories, News & Mentions about OPENSTREAM

Openstream’s Stories on HackerNoon