At Alloy, we work with consumer goods companies that make the products we eat, wear, and use every day, as well as the ones we occasionally splurge on

Alloy

Scientific American, Vol. XXXVII.—No. 2. [New Series.], July 14, 1877 by Various, is part of the HackerNoon Books Series. You can jump to any chapter in this book . An Alloy of Tin and Phosphorus. here An Alloy of Tin and Phosphorus. At the Graupen Tin Works, in Bohemia, an alloy of tin and phosphorus is made containing the greatest possible quantity of phosphorus which the tin is able to retain without losing any of it upon repeated meltings. This compound, which is neither entitled to the name of alloy nor is it a phosphide of tin, is employed in the manufacture of phosphorus-bronze. In the manufacture of phosphorus-bronze, by alloying copper with phosphorus-tin, no other precautions require to be observed than in the preparation of common bronze. As the different properties of phosphorus-bronze depend upon the proportions of phosphorus and of tin, two kinds of phosphorus-tin are prepared. No. 0 contains 5 per cent., and No. 1, 2½ per cent. of phosphorus. These two kinds suffice to make the greater part of all the desired mixtures. For special purposes, the Graupen Works make to order phosphorus-tin with any desired quantity of phosphorus not exceeding 5 per cent., which is the highest possible limit. It is claimed that phosphorus-bronze may be manufactured by the use of this phosphorus-tin as much as 40 per cent. cheaper than that now in the market, while it will only cost 8 per cent. more than the ordinary tin and copper bronze. No details are given of the method employed to make the phosphorus combine with tin, but the low melting point of tin as compared with that of copper would indicate that this would lead to the great saving promised above. About HackerNoon Book Series: We bring you the most important technical, scientific, and insightful public domain books. This book is part of the public domain. Various (2012). Scientific American, Vol. XXXVII. —No. 2. [New Series.], July 14, 1877. Urbana, Illinois: Project Gutenberg. Retrieved https://www.gutenberg.org/cache/epub/38481/pg38481-images.html This eBook is for the use of anyone anywhere at no cost and with almost no restrictions whatsoever. You may copy it, give it away or re-use it under the terms of the Project Gutenberg License included with this eBook or online at , located at . www.gutenberg.org https://www.gutenberg.org/policy/license.html

An Alloy of Tin and Phosphorus

The alloy is composed of copper, tin, spelter, or zinc and lead, which metals are manipulated. Scientific American, Vol. XXXVII.—No. 2. [New Series.], July 14, 1877 by Various, is part of the HackerNoon Books Series. You can jump to any chapter in this book . New Alloy. here New Alloy. A very beautiful new alloy, intended to replace brass in various ornamental uses, especially in window and door furniture, has been invented by W. A. Hopkins, of Paris. The alloy is composed of copper, tin, spelter, or zinc and lead, which metals are manipulated. A crucible is placed in the furnace and fired to red heat, and into the crucible thus heated the metals are placed in the proportions of—tin 1⅛ (say) 1 oz., spelter or zinc ½ oz., lead 5⁄16 of an ounce. These are the proportions he prefers to use, as he has found them to give excellent and satisfactory results, but he does not intend to confine himself rigidly to the precise proportions named, as they may, perhaps, be slightly varied in some particulars without materially detracting from the beautiful color of the alloy which it is intended to produce. The molten metals are kept well stirred, and any impurities therein should be removed. When thoroughly mixed, this alloy, which is termed the first alloy, is poured off into ingot moulds and left to cool. Copper, in the proportion of eight parts to one of this first alloy, is then placed in the crucible and brought to melting heat, when the tin or first alloy is added and intimately mixed with the copper, for which purpose the molten mass must be well stirred for several minutes; it is then poured into ingot moulds for sale in the form of ingots, or it may be poured into pattern moulds so as to produce the articles required. This is the mode of manipulation which it is preferred to employ, as an opportunity is thus afforded of removing any impurities from the first alloy before mixing it with the copper; but all the metals may, if preferred, be mixed together in the proportions given and melted at one operation. By this means an alloy is obtained of great strength, and of a very beautiful appearance, and which is particularly suitable for small work, such, for instance, as window and door furniture and other house furniture which is usually made in brass or other alloy of copper, though it is not intended to confine its use to such articles. About HackerNoon Book Series: We bring you the most important technical, scientific, and insightful public domain books. This book is part of the public domain. Various (2012). Scientific American, Vol. XXXVII. —No. 2. [New Series.], July 14, 1877. Urbana, Illinois: Project Gutenberg. Retrieved https://www.gutenberg.org/cache/epub/38481/pg38481-images.html This eBook is for the use of anyone anywhere at no cost and with almost no restrictions whatsoever. You may copy it, give it away or re-use it under the terms of the Project Gutenberg License included with this eBook or online at , located at . www.gutenberg.org https://www.gutenberg.org/policy/license.html

A very beautiful new alloy

ne of the more exciting announcements at this year was the preview of , a fully-managed, PostgreSQL-compatible database service on Google Cloud. O Google I/O AlloyDB for PostgreSQL The AlloyDB team made bold claims about their service: Compared to standard PostgreSQL, AlloyDB is more than , and up to . 4x faster for transactional workloads 100x faster for analytical workloads Compared to Amazon's service (they don't say but we can guess it's Aurora 🤔), AlloyDB is . 2x faster for transactional workloads These are impressive numbers! How were the AlloyDB developers able to achieve this kind of acceleration for both transactional and analytical workloads? What architectural innovations were necessary in order to get there? And are there lessons to be learned for architects and developers of similar cloud-scale distributed systems? In this post, I'll walk you through the architecture and design details of AlloyDB for PostgreSQL, focussing particularly on its storage engine, which is the secret sauce behind these performance numbers and is also the catalyst for a number of administrative and management benefits of the service. Let's begin, shall we? AlloyDB service description and benefits Before we go down the intriguing rabbit hole of design choices, trade-offs, and cost implications of AlloyDB's architecture, let's briefly put on our Product Management hat 🧢, to define what AlloyDB is, and to articulate how enterprises can expect to benefit from it. This will help set the context for the rest of the discussion. First off, AlloyDB is a PostgreSQL-compatible database. In theory at least, what this means is that a customer can simply migrate data from their existing PostgreSQL database to AlloyDB, update the database connection strings in their applications to point to the AlloyDB instance, and it would all work just fine! This is good as it eases the adoption process in the customers' minds. A quick aside: Notice how varied databases now are all PostgreSQL-compatible. This is an exciting trend in my opinion! For the longest time, the application world was crippled due to a lack of protocol and grammar standardization. PostgreSQL, MySQL, and other databases may seem similar on the surface but are worlds apart when it comes to the client-server protocols they use, or the SQL grammars they support. It's heartening to see CockroachDB, Redshift, Aurora, and now AlloyDB, all converging on PostgreSQL. This will vastly simplify driver development, application coding, and security and observability tooling! Figure 1 shows the high level functional components of AlloyDB. How would we describe this service? Firstly, AlloyDB is a regional service (think , , etc. in AWS lingo), which is to say it is multi-region. us-east-1 us-west-1 not Secondly, within a region, AlloyDB runs across multiple availability zones (AZs), or data centers, for redundancy and availability reasons. Thus, it multi-zoned. is Lastly, the service comprises multiple compute instances to which applications can connect. There's the singleton primary instance for servicing read and write operations. And then, there are multiple replica instances for servicing-only read operations. Replica instances service write operations. Notably, the multi-AZ setup allows a replica to be promoted to the primary status if the existing primary instance were to die for some reason. do not Also, since AlloyDB is a fully-managed service, server administration, patching, data protection, and replication are offloaded to the cloud provider. These operational and management benefits, combined with AlloyDB's performance and scalability properties, would make AlloyDB a compelling product for enterprises that might be looking to make the move to the cloud. With the PM-speak now out of the way, let's dive into the architectural details. Monolithic database design limitations Here's a question to ponder: Does anything about Figure 1 above strike you as odd? If you've studied the architectures of the standard PostgreSQL or MySQL databases, you'd know that their designs are in nature. What this means is that the database runs on a single node or machine, on which both compute (CPU) and storage (disks) are collocated. monolithic Such databases have a fundamental but well-documented design issue. Due to the strong coupling between compute and storage, as data volumes increase, one cannot arbitrarily keep on adding more disks to the machine. Similarly, as the number or diversity of the workloads increase, more compute (CPUs) cannot be simply added except through a re-provisioning of the machine. Often, the only solution is to scale-up i.e., switch up to more and more powerful machines. This is both expensive and naturally limited by the constraints imposed by Moore's Law. For these reasons, it's not common to see these monolithic designs being run at cloud-scales of either the data volumes or workloads. Herein lies the crux of AlloyDB's innovation and architectural differentiation, and the AlloyDB team calls it ! Disaggregation of Compute and Storage Another quick aside: The word "disaggregation" seems fancy but it's really what the rest of the data warehouse community calls "decoupling", and the concept has been around for more than a decade now. Snowflake was one of the first products, whose architecture was built on compute decoupled from storage, which it achieved by storing data in S3 buckets, and running the database engine on EC2 instances. What does this decoupling, or disaggregation, of compute from storage mean, and what benefits does it provide? For database systems running business-critical transactional or analytical workloads, it's extremely important to be able to scale compute and storage independently. Not being able to do so leads to a number of operational and administrative challenges at scale: Compute resources cannot be tailored to dynamically adapt to the demands of diverse concurrent workloads without also impacting storage and vice versa. Workloads interfere with each other when they run on such architectures since they all vie for the same shared resources. This leads to unpredictable performance overall. In multi-node primary-replica setups, there's a considerable lag of data replication between the primary and replica instances. Based on the database load, failover times may end up being longer than acceptable, and also less predictable. Decoupling computing from storage solves or mitigates, many of these limitations. It allows provisioning and scaling of computing according to each workload running in the database without impacting either the storage or the performance of other workloads. When needed, newer read replicas can be spun up dynamically to improve the performance of reading workloads. Through efficient replication between the primary and the replicas, failover times can be improved and made more predictable. How's all this achieved, though? The answer lies in the design of AlloyDB's storage subsystem. Let's look at that next! AlloyDB storage layer components Figure 2 below opens up AlloyDB's storage layer from Figure 1 to reveal its components. The storage layer has three major services: : A low-latency service for fast writing of WAL (write-ahead log) records. Log storage service : Processes WAL records asynchronously to generate materialized up-to-date data blocks. Log processing service : Stores data blocks durably across availability zones for fault-tolerance with support for sharding. Block storage service You may be wondering what a WAL is and how it's different from data blocks. Imagine a database table in a banking application. Each credit or debit transaction on an account updates the account's balance. At the storage layer, however, data is never updated in place due to a number of reasons pertaining to failure handling, recovery, and performance. Instead, databases will typically "write-ahead", or append, to a log file, the details of each transaction is committed. Balances Examples of transaction details written to the WAL include the statement type (INSERT, UPDATE, DELETE), the affected table columns, the new data values, and so on. The WAL has an added benefit in that the database's write behavior is forced to be strictly append-only, leading to a good performance at the storage layer, which is good at handling appends (also known as streaming writes) far better than random writes. Our banking application will, thus, append a new record to the WAL for each successful credit or debit transaction. Obviously, this leads to the question, how are reads serviced? In order to keep reads snappy and fast, a process called "log replay" reconstructs the data by replaying the WAL records in chronological order, and storing them as blocks in the database. Thus, we can think of data blocks are the most up-to-date state of the database, while a WAL is a record of all the state transitions the database has gone through. With this context, let's now revisit each storage component's responsibilities in Figure 2: The is responsible for managing the WAL, and keeping writes fast and snappy. log storage service The is responsible for replaying the WAL records to create up-to-date snapshots of data blocks, and sending them to the block storage service. It is also responsible for serving these data blocks to the primary and replica database instances upon demand during reads. log processing service The is responsible for ensuring data blocks are promptly and reliably persisted locally and replicated across all zones region-wide. It also feeds the with the relevant data blocks whenever needed. block storage service log processing service By the way, an important detail to remember is that the entire storage subsystem is . What this means is that there's at least one instance of each service running in each of the availability zones within the region for high availability and fault-tolerance. multi-zoned So yeah, we have decoupled compute from storage and added a ton of complexity in the process! 🤯 To what end, though? What benefits does this architecture provide? Due to the decoupling, the services can be scaled individually to accommodate increased demand. For example, even though the is shown as a singleton, multiple instances of it can be spun up dynamically within each availability zone to speed up WAL processing, or to serve data blocks at a higher throughput for data-intensive workloads. Similarly, the can be scaled to meet the demands of durability and reliability. log processing service block storage service Tasks such as data replication and data backups can be delegated completely to the storage layer without impacting the compute instances (which function purely as query optimization and execution engines). New replica creation and failure recovery operations are sped up because a) the most up-to-date data blocks are already available (having been reconstructed from the WAL), and b) data blocks can be read from any of the availability zones, no matter whether it's an infrastructure failure or a zone failure. AlloyDB writes processing We'll now do another double-click of AlloyDB's architecture to examine how it handles writes. Figure 3 is essentially a reproduction of Figure 2 with the read paths removed so that we can just focus on the write paths. When a client application wants to issue a write, such as an INSERT, UPDATE, or DELETE statement, it must always do so against the primary instance since replica instances only serve reads. When the primary instance receives a write, it appends a new record to the WAL upon successful execution of the statement and a transaction commit (as I explained before). As soon as the acknowledges that the WAL records has been durably persisted, the primary instance responds to the client application to indicate its write was successful. log storage service Simultaneously, the primary instance also transmits the WAL record to all active replicas so that they can update their respective internal states. This is a critical step in the write workflow, as otherwise the replicas could quickly go out of sync from the primary, and start serving stale data to clients. Asynchronously, the consumes the newly written WAL records, replays them on the current state of the relevant data blocks, and persists them to the underlying storage service. log processing service Of course, there are a number of gory implementation details relating to MVCC and ACID semantics. Still, this would be a fair and accurate high-level representation of what goes on under the hood for processing writes. AlloyDB reads processing Next, we'll look at how AlloyDB's architecture processes read. Figure 4 below shows the data paths for reads similar to how Figure 3 above showed the paths for writes exclusively. Unlike for writes, the client application is free to connect to any of the instances to issue a read operation (a SELECT statement, for example). Somewhat surprisingly, though, the processing for reads is far more involved than that for writing. By the way, all this time, when we were extolling the virtues of decoupling compute from storage, we didn't bring up an obvious but undesirable side-effect of it. When compute and storage are decoupled, the read and write throughputs suffer, due to additional network hops, transport protocol overheads, and such, between the compute and storage services. In order to compensate for these new overheads, all architectures built on the notion of computer-storage decoupling resort to aggressive caching strategies, sometimes at multiple layers, to improve read performance. AlloyDB is no different! As seen in Figure 4, in addition to the database's own buffer cache, the AlloyDB architects have added what they call an "ultra-fast cache" to the compute instances. This is in order to both extend the working set size, and mitigate any slowdown due to network roundtrips due to missing data blocks. Also noteworthy is the fact that the missing data blocks are fetched from the , and not the . The , in addition to having the ability to process the WAL also supports PostgreSQL's buffer cache interface. It does this in order to cache data blocks from the and serve them to the primary and replica instances. The , thus, effectively becomes another caching layer in the architecture. log processing service block storage service log processing service block storage service log processing service To draw a very rough analogy  (due apologies to the purists 😜) with microprocessor caches: Think of the buffer cache in the primary/replica instances as the L1 cache Think of the ultra-fast cache as the L2 cache Think of the buffer cache as the L3 cache log processing service See what I mean? The caching strategy employed in AlloyDB's architecture is pretty generous and aggressive. This alone, combined with the parallelism delivered by multiple replica instances, likely accounts for the impressive performance numbers quoted in AlloyDB's preview announcement. As you must be rightly thinking, this would have a considerable effect on the provisioning and operating costs! Enterprises, though, who are looking to move to the cloud for its benefits, would likely not mind paying up! o, we've looked at AlloyDB's architecture to understand how it disaggregates/decouples compute instances from the storage layer, and how it processes reads and writes. We examined the design choices that were made to ensure reads and writes remain highly performant despite the decoupled architecture. AlloyDB uses caching generously, which explains the impressive performance numbers for both transactional and analytical workloads, compared to standard PostgreSQL and Amazon's Aurora. S AlloyDB seems very well designed, and I like it a lot! Congratulations to their engineering team! Also Published here References Introducing AlloyDB for PostgreSQL: Free yourself from expensive, legacy databases AlloyDB for PostgreSQL under the hood: Intelligent, database-aware storage

Taking a Closer Look into Google's AlloyDB Architecture for PostgreSQL

Major changes ahead This week marks the first year anniversary since the of , and to celebrate it, we are releasing the v0.5 . With it we get closer to the much coveted v1.0 release, when AlloyCI will be fully ready (and fully supported) for production environments. first commit AlloyCI version This version of AlloyCI comes with a wide variety of changes. The most noticeable being the new and improved design, and support for GitHub Enterprise. Design Changes For the past year we have been using CoreUI as the basis of the design. At the beginning, it seemed like a good “admin” template that offered a lot of functionality. Unfortunately it started to feel dated, and bloated pretty quickly. The way it was used with AlloyCI was not ideal and it was not nice to look at. We decided it was time for a change. The new design feels clean, modern, and is easy to look at. Some elements remained, like the use of cards to represent the projects, and the use of panels to separate related information: New projects view The use of colors to represent statuses also stayed, but with a new pantone: New pipelines view (single project) The builds view was completely reworked to better illustrate the different stages of a pipeline, and to give a quick overview of the status of each build within that pipeline: New builds view (single pipeline) The single build view also suffered major changes. Now each build has its own page, with dedicated, always visible information about the build on an aside to the left, and the entire build output (which will automatically update if the build is running) to the the right. New build view The project settings view did not change that much, but it still looks cleaner and now groups related information more tightly: Project settings view New features GitHub Enterprise This new version of AlloyCI also comes with a host of new features. The biggest one for this release is support for GitHub released version 2.13 a few days ago with support for GitHub Apps. This meant that AlloyCI would finally be compatible with GHE. After a few changes to the way the system communicates with GitHub, mainly making endpoints configurable, full compatibility was achieved. GitHub Enterprise. Builds Builds can now be restarted individually, without having to restart the entire pipeline. The old build data will be kept, and a new job will be created for the new build. Build Artifacts The other major new features revolve around build artifacts. With this release we added: Artifacts can be managed and downloaded from the build view Expired artifacts will be pruned once a day (or at any configurable interval) Artifacts can be kept forever, if the user chooses to Bug fixes A couple of bugs have also been fixed with this release: When cancelling a pipeline, all builds were marked as cancelled, now only builds that are either , , or are updated. created running pending After the upgrade to Guardian 1.0, it became impossible to add another authentication method to an existing user. This has been fixed. This is pretty much it for this release. The roadmap ahead includes adding S3 bucket support for build artifacts, so that they can be uploaded to services like AmazonS3, or Digital Ocean Spaces. With this support implemented, it will become possible for AlloyCI installations that are hosted on Heroku to also make use of build artifacts, something that is not possible right now. Until next time! ❤️

AlloyCI v0.5 released!

Project management with GitHub Issues & Boards Photo by on Todd Quackenbush Unsplash This blog post is part of a series of posts detailing the process of AlloyCI. The previous entries have been: development Part 1: Starting a new Project Part 2: Structuring a new Phoenix Project Now that we have started a new project, we most certainly have a lot on our minds about what needs to be done. We might already know where to start, what comes next, and how to get there, or we might not. If we already know what we want, it makes perfect sense to write it down, give it form, and expand on the ideas and use cases we want to tackle. If we don’t, then writing down the goals we want to accomplish will most likely light the way. Once we have written the main goal, the next step would be to break it down into smaller goals, and the those goals into smaller tasks that we can accomplish easily. In both cases, we need a way to keep track of what we need to work on. For a software project, issues are a great tool for this. GitHub Organization You can create an issue for every task that needs to be tackled, keep them, organized using labels, e.g. have a label for User related, or front-end related tasks, and to keep a record of your progress. AlloyCI’s Issue Labels I have been using them for AlloyCI since early on. The project has different labels to organize different issues, and filter them. Even though, for now, it’s only me working on AlloyCI, having all the tasks that need tackling, and also a history of what has already been done, will really help anyone wanting to get started with the project. It might seem tedious if it’s only you working on a project to devote so much time to issues, but it will really help. You will most likely be the one that benefits the most once you come back after a few months of not working on the project and see the list of thing to be done. If you hadn’t written them down, you would have most likely forgotten about this tiny feature you wanted to add, that seems unimportant but greatly increases user’s happiness. Planning If you want to go a step further and keep track of what issues are being worked on, what their statuses are, and what the next steps are, you can use GitHub’s Issue Boards. The Boards allow you to create columns that represent stages of the development process, like “In Progress”, “Waiting Review”, “To Be Deployed”, etc. They give you an instant visual overview of how the project is progressing. AlloyCI’s Issue Board Setting goals To keep myself motivated I also added milestones to the issues. The first milestone was v0.1, in other words, the minimum amount of functionality needed in order for AlloyCI to actually be a CI coordinator. I set a date for it, as a tentative goal for when I wanted to be done with the first part. I wasn’t able to keep it (I pushed it back at least 2 times and by more than 2 months), but it gave me something to aim for. AlloyCI’s current milestones For v1.0 I haven’t set a date, it is an open milestone, because there are some pretty complex issues assigned to it, but I still look at it as a goal. That is where I want the to project to be. It can be in a few months, in can be next year, but I I’m going to get there. know There are many tools to help you with project management, but for a small team (or a team of one) the tools GitHub gives you for free are more than enough to get you where you need to be.

AlloyCI Dev Diary — Part 3

Structuring a new Phoenix Project Photo by on William Bout Unsplash This blog post is part of a series of posts detailing the process of . The previous entries have been: development AlloyCI Part 1: Starting a new Project If you are coming from a Rails background, you might find the folder structure of a Phoenix project a bit weird, specially after the changes in 1.3. But don’t be put off by this, the structure actually makes a lot of sense, specially the new 1.3 structure. Unlike Rails, there is no folder where most of your code will live. Phoenix 1.3 projects follow the structure of pretty much any regular Elixir project, this means that most of your code will live inside the folder. app/ lib/my_app As you might now, Phoenix 1.3 changed the way projects are structured. Before this change most of your code would be under , with little to no further organizations. You would have a directory, where you would put all your business logic, without much consideration as to how they interact together. This folder would get pretty large, pretty fast. Also the name seems to imply an object, but in Elixir, there are no objects, so storing your data access layer under a folder named makes little contextual sense. web/ models model models Data & Contexts Phoenix 1.3, by default, guides you towards a better way of organizing your code. Controllers, Views, Templates go under , database migrations and related files go under , and the rest of your Elixir code will live under . This is the basic setup, and you can tweak it, and change it as much as you like. You have complete liberty as to how to organize your code. lib/my_app_web priv/repo lib/my_app Since I started writing AlloyCI Phoenix 1.3 was fully released, some of the folder structure is different than the one the latest 1.3 generators create. I prefer the way AlloyCI is structured right now, because I really don’t like the way they renamed the folder to and everything inside it from to . I really prefer the separation in the module name, and the fact that the app name is not repeated in the web folder name. Thanks to the flexibility Phoenix provides, I don’t need to follow the conventions, though. before web/ alloy_ci_web/ AlloyCi.Web.XXX AlloyCiWeb.XXX Anyways, the most important part about the structure changes, is that Phoenix now guides you towards using for structuring your data access layer. contexts Using AlloyCI as an example, we have the context (which is under folder), where the , , and schemas live. These 3 schemas are closely related, and belong to the same business logic, namely the handling of Accounts. Accounts lib/alloy_ci/accounts User Authentication Installation If you look closely at the under the folder, you will see that there are no functions in the schema files, other than the changeset function. This means that I would need to either go straight through to manipulate the database data (not recommended) or that I need an API boundary that will let me perform actions on the accounts related schemas. files accounts Ecto This is where the module comes into play. This module is the boundary with which AlloyCI will interact if it needs to perform an action on any of the schemas related to an Account. All public functions of this module provide an easy way to manipulate the data, while providing security through a layer of indirection. AlloyCi.Accounts This is the purpose of They provide a boundary between your data layer and your business logic, and allow you to have an explicit contract that tells you how you can manipulate your data. It also allows you to stay independent from . contexts. Ecto Let’s say, in the future, you’d like to switch from to the “latest, coolestDB driver”. If you didn’t use an abstraction layer, like the contexts, you would have to refactor every function across the codebase that used to communicate to the data layer. since we are using contexts, we would only need to refactor the code inside the context itself. Ecto Ecto But Data Presentation The code that will actually present your data to the user can live under the or folders, depending on how you want to structure it (the automatic generator will default to but I prefer the former). lib/my_app/web lib/my_app_web lib/my_app_web In here you will find the folders where your controllers, views, templates and channels will live. Let’s start with the presentation layer. Views & Templates If you come from a Rails background, you might wonder why there are two components to presenting the data, when in Rails all you need is the folder. In Phoenix, the “views” are not composed of templated HTML files, but rather they are regular Elixir modules. These modules are there to help you share code with the template, and fulfill a similar purpose as the Rails “View Helpers”, but are, by default, specific to a single controller (other views are not loaded, unlike Rails that loads all view helpers, regardless of the controller being called). This separation makes it easier to use the same signature on similar helper functions needed to present data (without really overloading them), depending on which controller is being called, thus simplifying your code. views The templates are, then, where your HTML code lives. The template files are saved as files (meaning embedded Elixir), and are very similar to files. The syntax is exactly the same, but instead of Ruby code inside, you write Elixir code 😄 *.html.eex erb A very important distinction between Phoenix and Rails is how you share information between the controller and the template. In Rails, it is enough to declare an instance variable with and it will be available to the template/view. @something Given the functional nature of Elixir, in Phoenix you need to explicitly pass the information you wish to be available to the views in the function. These are called assigns. As an example, here is the action of the : render show PipelineController Everything that comes after are the assigns, so the variables available to the templates related to the action are , , and . We can see an example of how to use them in this snippet from the pipeline info header: "show.html" show builds pipeline current_user Once a variable has been assigned, it is available to the template via , just like with Rails. Functions defined the view file of the same name as the controller (in this example ) are immediately available to the template. In the above example, creates an HTML link to the specific commit on GitHub. @var_name pipeline_view.ex sha_link/1 Controllers In structure, Phoenix Controllers are very similar to Rails Controllers, with the main difference being described above. When generated by the helper tools, they will the same index, show, edit, update, and delete actions as their Rails counterparts. And just as with Rails Controllers, you can define any action you desire by defining a function, and connecting a route to it. Channels Phoenix Channels are used to communicate with the web client via Web Sockets. They are similar to in Rails, but in my opinion, much more powerful, and . In AlloyCI, they are used to push the output of the build logs in real time, and to receive a pre formatted piece of HTML code to show the user’s repositories (more on how AlloyCI uses Channels will be discussed in another post). ActionCable performant And there you have it. That is the basic structure of a Phoenix project. There are other components that we haven’t covered here, like , or . We will discuss these advanced topics in a future blog post. Plugs Background Jobs

AlloyCI Dev Diary — Part 2

Starting a new project Going by AlloyCI’s , you would think that the idea to write a CI coordinator came pretty naturally for me, but it actually took me quite some time to find the perfect for a new project. introduction post idea Since the main purpose of starting this new project was to cement my newly acquired Elixir knowledge, I could have chosen something simpler, or even just gone along with the tutorial projects found in all language learning books. Something like a blog management system, a issue API client, a very simple CMS, etc. But I knew that if I went that route, I would stop adding features to the project as soon as I was done with the book. I did not want that to happen. GitHub I really think that a longer lived, more useful project, will have a more positive effect, than a one-off tutorial project, specially if you try to make something other people might find useful. Coming up with something other people might find useful is ! But here’s the catch: really hard A good starting point is finding a problem that you might have that isn’t quite solved by current offerings. Chances are that more than one person out there has the same problem as you. Now, I’m not going to lie, it is going to take you of time to come up with something usable, but that is part of the fun! a lot I took me around 3 months until the CI coordinator idea creeped into my head. Before that I thought about re-implementing the Ruby library in Elixir, but that idea quickly died when I realized that SAML support is almost non-existing in Erlang, let alone Elixir, at least nothing that can compare with and it’s power. Once that idea died, I continued with the re-implementing something idea for a while. I eventually gave up when I noticed that every idea I had to re-implement something, already had a corresponding Elixir project. In hindsight this might not have been the best idea. Even if there is already a library out there, you can always learn a lot by re-implementing it. Your own particular views and opinions can lead you to a better implementation of an existing solution. At the very least you will learn a lot. omniauth-saml ruby-saml The idea to create the CI coordinator came to me, as many ideas often do, while I was in the shower. Clearing your mind and not thinking about a problem is often the best way to find a solution, I have experienced this multiple times in my life. Once the idea “presented itself”, I knew it had great potential. By re-implementing the GitLab CI API in Elixir I would be decoupling its system and liberating the GitLab Runner, making it compatible with other SCM systems. It would give new possibilities to an already great, but locked down project (the CI Runner). The last part was what really got me excited to get started. I could leverage an existing project to build something new, and simplify some of my approach, which would give me a head start, and make it possible to release an Alpha version quicker. The next day I got started with the basic project layout and did not go to bed until 4 am of the next day 😬 And that is the squick story about how I got started with AlloyCI. of the Dev Diary will be about project structure for a new Phoenix project, and the transition between 1.2 and 1.3 that AlloyCI went through before its public release. Part 2

AlloyCI Dev Diary — Part 1

A new Continuous Integration Coordinator written in Elixir & Phoenix I have been wanting to learn Elixir and for a while now, and a couple of months ago I finally took the plunge. I started reading “Programming 1.3” by Dave Thomas, and “Programming Phoenix” by Chris McCord, et. al. (both from “The Pragmatic Bookshelf” and highly recommended if you want to get started with Elixir and Phoenix). Phoenix Elixir I followed most of the code guides in the books, but it still felt kind of foreign sometimes. I wanted to build something a bit more complex, and challenging. I also wanted to release it as open source software, so it had to be useful to a wider audience than just me. I’ve always liked CI systems and the value they provide, but I have yet to find one that offers everything I need. I want it to be highly configurable, easily scalable, easy to install, with a robust executor that can run on multiple operating systems, and to be . The CI service that comes closest to this is GitLab CI, but it is tightly coupled with GitLab itself, so I cannot use it with GitHub. I also don’t want to install a whole Source Code Management system for it’s CI service. open source just That is where the idea to create a CI system came from. I learned a lot about how CI systems should work from my time at GitLab, so it felt like the right step forward. In fact, I decided to use the GitLab CI Runner as the executor for this project. The GitLab CI Runner is written in and is the piece of code that actually takes care of running the builds, and whatever other action you need in order to execute your pipeline. The greatest thing about it, is that it is basically a , meaning that all you need to do is tell it what to do in the way it expects, and it will do it! It is also incredibly powerful and configurable, it can use Docker to execute your builds, and takes advantage of Docker Machine to let you autoscale the machines it uses for your builds, it can use Kubernetes, run on top of Digital Ocean’s VMs, run Xcode and Windows builds, and a lot more. Go “dumb box” Using an already existing project as the executor for my CI system allowed me to concentrate on the coordinator, which is the piece of software that reads your project’s configuration, creates the pipelines, and builds, and sends all this information to the executor in the way it expects it to be. It also collects all the information relayed by the executor, like build trace and other artifacts, and presents the to the user in an easily digestible manner. Since this is an introduction post, I will not go into details of the implementation of the different parts of AlloyCI. For that I will create and will go into as much detail as possible. For now, let me tell you about its features. several separate posts (Part 1 of the ) AlloyCI Dev Diary is live! Features Please keep in mind that this is the first public release of AlloyCI, and it is only at version 0.1.0. A lot of the features will improve over time and a lot more will come as we move towards 1.0. You can check the roadmap to 1.0 here: and if you’d like to contribute anything, please go ahead. https://github.com/AlloyCI/alloy_ci/projects/1 Basic CI functionality For version 0.1 the goal was to have at least the very basics of CI functionality. These include the following: Parse a configuration file to create and prepare the builds and pipelines Support different build stages Allow builds and runners to be tagged, thus allowing for fine grained control of which runner picks up which build Process and send the build information to the GitLab Runner in the expected format Accept information received from the GitLab Runner about the specific build Show the output trace of each build and follow it when looking at an active build Properly manage the status of each build, and send to the Runner only builds that should be processed Report to GitHub the status of each push via the reference of its last commit SHA Send notifications to the user about the status of a pipeline via email and/or Slack Perform auto-deploys via the stage deploy As commercially available CI systems go, AlloyCI is dwarfed by the plethora of features other CI services offer, but that’s OK. I wanted to start simple and distill the CI experience to only its most necessary features. Bells and whistles will come later 😉 Extras Something I did add just for the “nice to have” value are build statistics per project and runner. You can see some nice charts portraying the amount of build in the past week, how many have succeeded or failed, and the total of builds processed. Project’s Settings The runners chart will also brake down the amount of builds processed by project, so you can see which runners are picking up which builds. Runner’s Admin Settings How can you try it out? You can install AlloyCI on your own servers using Docker, and Docker Compose. Please follow our to get started. documentation The easiest way to try AlloyCI out is by going to , logging in with your GitHub account, and adding a project. https://alloy-ci.com New Project’s View Before being able to add a project, you will be prompted to install the AlloyCI integration on your account. After doing this, the project you’ll want to add needs to have an configuration file before it can be added. You can read more about how to create this file . Here is a quick example for Elixir: .alloy-ci.json here {"image": "elixir:latest","services": ["postgres"],"before_script": ["mix deps.get"],"after_script": ["rm secrets"],"stages": ["build","test","deploy"],"mix credo": {"stage": "build","script": ["mix credo"],"tags": ["elixir"]},"mix test": {"stage": "test","script": ["mix test"],"tags": ["postgres", "elixir"]}} Once the file has been committed to your repository, you are ready to add it to AlloyCI. After the project has been added, and new code has been pushed, you will see the newly created pipeline in the project’s dashboard. Project’s Dashboard If you click on the pipeline, it will take you to the its builds, where you can follow their output and overall status. Build Job’s Output The test server has limited resources and only has 2 registered runners to process builds, so it will take a while to process your pipeline. Luckily you can register your own runner as a “Project Specific Runner”, which means it will only pick up builds from your own, particular project. Go to your project’s settings for instructions on how to add a specific runner. Why did I build AlloyCI? As I mentioned before, I built AlloyCI to learn Elixir and Phoenix. I chose a CI coordinator, because I’m familiar with them, and wanted to add my own take on them. I’m also releasing it as licensed open source code, because I think other people might find it useful. Having a non-trivial codebase to look at is a really great resource to learn a new language. I know my code is far from perfect, but it is a start, and I would love for people to look at it, criticize it, and provide useful feedback, so that I, , can learn and benefit from this project. GPLv3 and others I will continue to develop AlloyCI for quite some time. I plan to maintain it for the foreseeable future, and I might even consider writing a new runner/executor in Elixir itself, so that the project does not rely on GitLab’s good intentions of keeping the GitLab Runner open source and maintained. — If you like the project, please star it on GitHub, and if you think of something the project could use, or have some suggestion or feedback, by all means, please open an issue. I would ❤️ to hear what you think.

ALLOY

alloy.ai

-- employees

Since n.d.

COMPANY RANKING

ALLOY

EVERGREEN INDEX #1371

Representing this company? Click here to claim and customize this page!

Alloy WIKI

Apply to get your Tech Company News Page live on HackerNoon.com today!

ALLOY

alloy.ai

-- employees

Since n.d.

COMPANY RANKING

ALLOY

EVERGREEN INDEX #1371

RELATED COMPANIESRANK

STORIES ABOUT ALLOY

Representing this company? Click here to claim and customize this page!

Alloy WIKI

Apply to get your Tech Company News Page live on HackerNoon.com today!