Hackernoon logoRethinking Programming: Network-Aware Type System by@lafernando

Rethinking Programming: Network-Aware Type System

Author profile picture

@lafernandoAnjana Fernando

Software architect and evangelist; Director of Developer Relations @ WSO2 Inc.

Introduction

In a programming language, the type system gives you the foundation in representing data in your programs and implementing logic. It provides the means of creating abstractions to the solutions you are providing. Some programming languages will provide you the most basic functionality, while others will give you in-built functionality for specialized domains, such as data frames for data processing jobs, as found in languages such as Python and R, and vectors and complex numbers for numerical computing as seen in Matlab and Octave. 
But how about in the domain of networked-services programming? This is not an area that has been ventured into frequently. In the current development landscape, more and more functionality is available in the cloud. This transforms our implementation of business logic into mostly service integration through the network. Basically, our application logic and integration code are converging.
There will no longer be heavy-weight centralized ESBs intercepting all application requests to make integration decisions elsewhere, but rather, the endpoints themselves have got smarter. This inversion of responsibility provides much more power and control to the developer and ultimately results in increased productivity and reduced release cycle times. 
As the developer is provided with the added responsibility of working with networked-resources in their code, it is critical that the programming language itself aids in this operation. This will involve aspects such as network communication resiliency, discovery, data mapping, transformation, and more.
Having an effective type system can greatly help in the core operations of these features. Ballerina is a programming language that was built ground-up with these requirements in mind. It has a network-friendly type system that helps the developer a great deal in the domain of network-distributed programming. In the following sections, we will take a look at this in detail. 

Statically Typed and Structural

Ballerina is a statically typed language, which means, all the types of variables are resolved at compile-time. Languages such as Javascript and Python are dynamically typed, where its variables do not have a specific type assigned, but rather, any value can be assigned to the variables. Java and C++ are statically typed, where similar to Ballerina, its types are checked at compile-time and only compatible values can be assigned. Statically typed languages are generally more robust, easier to debug, and aids in creating better tooling for the language. This comes as a tradeoff between the flexibility for a developer vs safety. Ballerina has chosen the path of code safety and better productivity in the long run. 
Also, Ballerina’s type system is structural, as opposed to a nominal type system. This means the type compatibility is figured out by considering the structure of the value. In a nominal type system, this is bound to the name of the actual type. Languages such as Java, C++, and C# follow a nominal type system. 
A type in Ballerina works with an abstraction of a value called a shape. A shape basically ignores the storage identity of a value. This means it does not take into account the storage location of a value when it is compared with other values. Let’s understand this more using a record type in Ballerina. Records in Ballerina have storage identity, which means, the respective variables will store a reference to the value rather than storing the actual value in the variable.
This is comparable to references in Java, or pointers in C++. Basic simple types in Ballerina, such as
int
,
float
, and
boolean
, do not have a storage identity and their values are directly stored in the variables. 
type DoorState record {|
    boolean open;
    boolean locked;
|};
Let’s create some values of the above DoorState record type. 
DoorState v1 = { open: false, locked: true };
DoorState v2 = { open: false, locked: true };
DoorState v3 = { open: false, locked: true };
The three variables above —
v1
,
v2
, and
v3
— all represent a single state of the door being closed and locked. But nonetheless, we have created three different values where each of the variables is stored in three distinct memory references. In this manner, we can actually create an infinite number of values with the
DoorState
type.
If we ignore the storage identity of the above values, we simply get the representation of the data it has, which is
{ open: false, locked: true }
. This is a single shape of the type
DoorState
. In this way, there are four possible shapes for
DoorState
. These four shapes are represented in the values referenced in the following variables.
DoorState ds1 = { open: true, locked: true };
DoorState ds2 = { open: true, locked: false };
DoorState ds3 = { open: false, locked: true };
DoorState ds4 = { open: false, locked: false };
A type in Ballerina represents a set of the possible shapes it can have. Any value which belongs to either of the above four shapes will be considered to be of the type
DoorState
Figure 1: Set of shapes of the type
DoorState
Figure 1 above shows the set of elements that contain the shapes that define the type
DoorState
.  
In Ballerina, it follows a semantic subtyping mechanism. It is defined by means of shapes, where S is a subtype of T, if the shapes representing S are a subset of the shapes representing T. Let’s demonstrate this behavior with a few examples. 
The type
boolean
is a simple basic type in Ballerina without a storage identity so its values become equivalent to its shapes. Therefore the
boolean
type is defined as having two shapes
true
and
false
The boolean type’s shapes can be defined in set notation as
sb = { true, false }
This can be visualized as seen in Figure 2 below. 
Figure 2: Set of shapes of the type
boolean
Now, according to our subtyping rules, we can derive new types based on the
boolean
type by creating subsets of its shapes. For example, a new type we can create is
boolean_false
where its only supported shape/value would be false. The new type is shown in Figure 3 below. 
Figure 3: Shapes sets of types
boolean
and
boolean_false
The new type
boolean_false
can be defined in Ballerina code in the following manner: 
type boolean_false false;
Here, we are using the value
false
in defining the new type
boolean_false
. In a more practical scenario, we can provide multiple values as a union in defining new types. A type created with a single value is called a singleton type. This new type can be used in the code in the following way. 
boolean_false bv1 = false;
boolean bv2 = true;
bv2 = bv1;
Here, we defined a new variable
bv1
of type
boolean_false
, and for this, we can assign only the value
false
, since this is the only supported value in the type. We also declare a variable
bv2
of type
boolean
, and later on, we assign the value of
bv1
to
bv2
. This assignment is possible because the
bv1
’s type is a subtype of
bv2
’s type. In simple terms, all the values that can be held in the variable
bv1
can be held in the variable
bv2
, thus the assignment is possible. 
The example above is not really useful in a practical scenario, but it demonstrates the basic idea of subtyping. For a more real-life example, let’s create a new type for holding the day of a month. 
A regular
int
’s value space is between -9,223,372,036,854,775,808 and 9,223,372,036,854,775,807. So the set which represents the shapes/values contains all the integers between those numbers. The new type we create will contain integer values from 1 to 31. 
type day_of_month 1|2|3|4|5|6|7|8|9|10|11|12|13|14|15|16|17|18|19|20|21|22|23|24|25|26|27|28|29|30|31;
The Ballerina code above defines the new type
day_of_month
as a union type. 
A union type in Ballerina is created by using the syntax
 T1|T2
, this basically means, the type
T1|T2
represents the union of the value spaces of type
T1
and
T2
. In the
day_of_month
type above, it is defined as a union of 31 singleton types representing each day of the month. The following code snippet shows the usage of the new type. 
day_of_month dm1 = 10;
day_of_month dm2 = 14;
day_of_month dm3 = 31;
int dm4 = dm3;
Here, we see how the
day_of_month
type is used in defining variables that can only store values between 1 to 31. Also, it shows how a
day_of_month
value can be assigned to an
int
variable, where
day_of_month
is a subtype of
int
since its value space is a subset of
int
’s value space. 
We have now seen how Ballerina’s subtyping works in relation to simple types. Let’s take a look at creating subtypes of records by revisiting our
DoorState
scenario. Here, we are going to create a new type
EmergencyDoorState
, where the
locked
field has to always have the value
false
. The resultant types and their shapes can be seen below in Figure 4. 
Figure 4: Shapes sets of types
DoorState
and
EmergencyDoorState
The type definition of
EmergencyDoorState
type is shown below:
type EmergencyDoorState record {|
    boolean open;
    boolean_false locked = false;
|};
In the above type, we have modified the field
locked
to be of type
boolean_false
, which makes it only have the value
false
. Here, we have also made the usage of default values in records in Ballerina.
So, in this case, the user does not have to always provide the value of a field, but rather in the absence of an explicit value provided by the user, the default value mentioned here will be used.
In this manner, the type
EmergencyDoorState
can only have the shapes
{ open: true, locked: false }
and
{ open: false, locked: false }
. These two elements make it a subset of the
DoorState
shapes set, thus
EmergencyDoorState
is a subtype of
DoorState
The following code snippet shows a sample usage of the
EmergencyDoorState
type. 
EmergencyDoorState eds1 = { open: true };
DoorState eds2 = eds1;
io:println("Door - Open: ", eds2.open, " Locked: ", eds2.locked);

Benefits of a Structural Type System

Having a structural type system is mainly beneficial when you have multiple systems interacting with each other, and data exchange and type compatibility can be resolved easier in this way. Let’s dive into an example that shows this behavior using Ballerina’s integrated query functionality. 
type Result record {|
    string name;
    string college;
    int grade;
|};
...
Result[] results = from var person in persons
                     where lgrade > 75
                       let int lgrade = (grades[person.name] ?: 0),
                       string targetCollege = "Stanford"
                       select {
                         name: person.name,
                         college: targetCollege,
                         grade: lgrade
                       };
In the example above, we filter records from a list of records and create a new value using the
select
clause, where its structure is defined dynamically at that time and the values are created. These values are assigned to an array of
Result
records, which is possible because the generated values are structurally compatible with the
Result
record type. 
In situations such as above, a separate system from our core application may be generating values to be consumed by us. In these cases, instead of worrying about sharing the code for the data type definitions and so on, you can simply concentrate on the compatibility of the data in order to ensure interoperability. 

Open-by-Default

Ballerina’s open-by-default concept is tied around the Robustness Principle. Here, we should design network-aware programs to accept all of the data that is sent to you and make the best effort to understand it. But when you are sending data, make your best effort to conform to the standard protocols that were agreed upon beforehand. This strategy makes sure we have the best chance of interacting with different systems in a reliable manner. 
The main facilitator of this in the type system is the open record concept in Ballerina. So far we have looked at closed records. Let’s take a look at a record type to represent the details of a person.
type Ethnicity "Asian"|"White"|"African American"|"Native American/Alaskan Native"|"Pacific Islander"|"Native Hawaiian";
type Person record {
    string name;
    int birthYear;  
    boolean married = false;
    Ethnicity ethnicity?;
};
Here, the type
Person
is an open record type, defined with an inclusive-record-type-descriptor by using the “
{
“ and “
}
” delimiters. This is the default behavior when defining record types in Ballerina. An open record is not limited to the fields that are declared in the record type. But rather, we can set additional fields that are not explicitly mentioned in the type definition.
The earlier
DoorState
record type was defined explicitly as a closed record type with an exclusive-record-type-descriptor by using the “
{|
” and “
|}
” delimiters in the definition. Because it was a closed record, we were able to list out all the possible shapes in the
DoorState
type. If this type was defined as an open record, we would have an infinite number of shapes since
DoorState
values can have any arbitrary field set in the code.
The
Person
record type above has an optional field ethnicity. In a record type, a field name with the suffix “
?
” makes it an optional field. This means the field value of ethnicity of a
Person
record can be skipped without setting a value. Later on, this field can be accessed using the “
?.
” operator, which would return a value of type
Ethnicity?
, which is equivalent to the union type
Ethnicity|()
. In Ballerina, the nil value and the type is represented by
()
.
Let’s create a new type
Student
, which will be a subtype of the
Person
type.
type Student record {
    string name;
    int birthYear;  
    boolean married;
    Ethnicity ethnicity?;
    string college;
};
The
Student
type defined above has an extra field
college
of type string compared to the
Person
type. All the possible shapes in the
Student
type are included in the set of shapes in
Person
as well.
This is possible because the
Person
type is an open type, and its shapes can have the
string
field called
college
as well. By any chance, if we make the
Person
type a closed record,
Student
will no longer be a subtype of
Person
Sample usage of the above types is shown below:
public function main() {
   Student s1 = { name: "Tom", birthYear: 1990, married: false,
                  college: "Yale" };
   Student s2 = { name: "Anne", birthYear: 1988, married: true,
                  ethnicity: "White", college: "Harvard" };
   Person p1 = s1;
   Ethnicity? eth = p1?.ethnicity;
   if eth != () {
       io:println("P1's ethnicity: ", eth);
   } else {
       io:println("P1's ethnicity: N/A");
   }
   Person p2 = s2;
   eth = p2?.ethnicity;
   if eth != () {
       io:println("P2's ethnicity: ", eth);
   } else {
       io:println("P2's ethnicity: N/A");
   }
   io:println(p2);
}
$ ballerina run sample.bal 
P1's ethnicity: N/A
P2's ethnicity: White
name=Anne birthYear=1988 married=true ethnicity=White college=Harvard

Network Communication with Data Binding

In this section, we will take a look at how we will be using type system features for records in implementing network data binding operations with structural validation, data types handling, and payload passthrough operations. 
The functionality will be demonstrated using an HTTP service in Ballerina: 
http:Client asianRecordsDB = new("http://example.com/");
@http:ServiceConfig {
   basePath: "/"
}
service RecordService on new http:Listener(8080) {
   @http:ResourceConfig {
       path: "/record",
       body: "entry"
   }
   resource function process(http:Caller caller, http:Request request,
                             Person entry) returns error? {
       if entry?.ethnicity == "Asian" {
           io:println("Asian Record: ", entry);
           json jsonPayload = check json.constructFrom(entry);
           _ = check asianRecordsDB->post("/store",
                                          <@untainteD> jsonPayload);
       } else {
           io:println("Non-Asian Record: ", entry);
       }
       check caller->respond();
   }
}
$ ballerina run sample.bal 
[ballerina/http] started HTTP/WS listener 0.0.0.0:8080

Test Scenarios

Scenario 1
A request is sent without setting the
married
field since the record type defines the
married
field as having a default value. This value is set in the resultant record value. Also, the
ethnicity
field is not set since it is marked as optional.
Request:
curl -d '{ "name": "John Little",  "birthYear": 1855 }' 
http://localhost:8080/record
Terminal output: Non-Asian Record: name=John Little birthYear=1855 married=false
Scenario 2
A request is sent with a
string
value given for the integer field
birthYear
. The service validates the value for the field and the request fails. 
Request:
curl -d '{ "name": "John Little",  "birthYear": "1855 BC" }' 
http://localhost:8080/record
Response: data binding failed: error {ballerina/lang.typedesc}
ConversionError message='map<json>' value cannot be converted to 'Person'
Scenario 3
A request is sent with the optional
ethnicity
field also set. 
Request: curl -d '{ "name": "Sunil Perera",  "birthYear": 1950, "married": true, "ethnicity": "Asian" }' http://localhost:8080/record
Terminal output: Asian Record: name=Sunil Perera birthYear=1950 married=true ethnicity=Asian
Scenario 4
A request is sent with a non-existing value of the
Ethnicity
union type. This is validated by the service and the request fails. 
Request:
curl -d '{ "name": "Tim Kern",  "birthYear": 1995, "ethnicity": "Japanese", "country": "Japan", "zipcode": "98101" }' 
http://localhost:8080/record
Response: data binding failed: error {ballerina/lang.typedesc}
ConversionError message='map<json>' value cannot be converted to 'Person'
Scenario 5
A request is sent with additional fields not explicitly mentioned in the
Person
type. Since
Person
is an open record type, the service accepts these extra fields and these are available to be passed through to other systems, e.g. a forwarding service. 
Request: curl -d '{ "name": "Tim Kern",  "birthYear": 1995, "ethnicity": "Asian", "country": "Japan", "zipcode": "98101" }' http://localhost:8080/record
Terminal output: Asian Record: name=Tim Kern birthYear=1995 married=false ethnicity=Asian country=Japan zipcode=98101
The above executions demonstrate how the Ballerina type system works hand-in-hand with the network data handling functionality in providing an intuitive experience for developers. 

Summary

In this article, we looked at how the unique features of the Ballerina type system enable us to model network communication patterns in an intuitive way that enables maximum productivity for developers. The seamless integration with the data types and the context awareness allows Ballerina programs to do effective network data operations. 
For a more extensive analysis of the type system, refer to the Ballerina language specification
Various examples on using in-built data types, such as JSON/XML, and other network-based functionality can be found in Ballerina by Example pages. 

Tags

The Noonification banner

Subscribe to get your daily round-up of top tech stories!