Upbound is the industry’s first platform for building internal cloud platforms

Upbound

upbound

The Way We Live Now by Anthony Trollope is part of the HackerNoon Books Series. You can jump to any chapter in this book . RUBY RUGGLES OBEYS HER GRANDFATHER here RUBY RUGGLES OBEYS HER GRANDFATHER. The next day there was great surprise at Sheep's Acre farm, which communicated itself to the towns of Bungay and Beccles, and even affected the ordinary quiet life of Carbury Manor. Ruby Ruggles had gone away, and at about twelve o'clock in the day the old farmer became aware of the fact. She had started early, at about seven in the morning; but Ruggles himself had been out long before that, and had not condescended to ask for her when he returned to the house for his breakfast. There had been a bad scene up in the bedroom overnight, after John Crumb had left the farm. The old man in his anger had tried to expel the girl; but she had hung on to the bed-post and would not go; and he had been frightened, when the maid came up crying and screaming murder. "You'll be out o' this to-morrow as sure as my name's Dannel Ruggles," said the farmer panting for breath. But for the gin which he had taken he would hardly have struck her;—but he had struck her, and pulled her by the hair, and knocked her about;—and in the morning she took him at his word and was away. About twelve he heard from the servant girl that she had gone. She had packed a box and had started up the road carrying the box herself. "Grandfather says I'm to go, and I'm gone," she had said to the girl. At the first cottage she had got a boy to carry her box into Beccles, and to Beccles she had walked. For an hour or two Ruggles sat, quiet, within the house, telling himself that she might do as she pleased with herself,—that he was well rid of her, and that from henceforth he would trouble himself no more about her. But by degrees there came upon him a feeling half of compassion and half of fear, with perhaps some mixture of love, instigating him to make search for her. She had been the same to him as a child, and what would people say of him if he allowed her to depart from him after this fashion? Then he remembered his violence the night before, and the fact that the servant girl had heard if she had not seen it. He could not drop his responsibility in regard to Ruby, even if he would. So, as a first step, he sent in a message to John Crumb, at Bungay, to tell him that Ruby Ruggles had gone off with a box to Beccles. John Crumb went open-mouthed with the news to Joe Mixet, and all Bungay soon knew that Ruby Ruggles had run away. After sending his message to Crumb the old man still sat thinking, and at last made up his mind that he would go to his landlord. He held a part of his farm under Roger Carbury, and Roger Carbury would tell him what he ought to do. A great trouble had come upon him. He would fain have been quiet, but his conscience and his heart and his terrors all were at work together,—and he found that he could not eat his dinner. So he had out his cart and horse and drove himself off to Carbury Hall. It was past four when he started, and he found the squire seated on the terrace after an early dinner, and with him was Father Barham, the priest. The old man was shown at once round into the garden, and was not long in telling his story. There had been words between him and his granddaughter about her lover. Her lover had been accepted and had come to the farm to claim his bride. Ruby had behaved very badly. The old man made the most of Ruby's bad behaviour, and of course as little as possible of his own violence. But he did explain that there had been threats used when Ruby refused to take the man, and that Ruby had, this day, taken herself off. "I always thought it was settled they were to be man and wife," said Roger. "It was settled, squoire;—and he war to have five hun'erd pound down;—money as I'd saved myself. Drat the jade." "Didn't she like him, Daniel?" "She liked him well enough till she'd seed somebody else." Then old Daniel paused, and shook his head, and was evidently the owner of a secret. The squire got up and walked round the garden with him,—and then the secret was told. The farmer was of opinion that there was something between the girl and Sir Felix. Sir Felix some weeks since had been seen near the farm and on the same occasion Ruby had been observed at some little distance from the house with her best clothes on. "He's been so little here, Daniel," said the squire. "It goes as tinder and a spark o' fire, that does," said the farmer. "Girls like Ruby don't want no time to be wooed by one such as that, though they'll fall-lall with a man like John Crumb for years." "I suppose she's gone to London." "Don't know nothing of where she's gone, squoire;—only she have gone some'eres. May be it's Lowestoffe. There's lots of quality at Lowestoffe a' washing theyselves in the sea." Then they returned to the priest, who might be supposed to be cognisant of the guiles of the world and competent to give advice on such an occasion as this. "If she was one of our people," said Father Barham, "we should have her back quick enough." "Would ye now?" said Ruggles, wishing at the moment that he and all his family had been brought up as Roman Catholics. "I don't see how you would have more chance of catching her than we have," said Carbury. "She'd catch herself. Wherever she might be she'd go to the priest, and he wouldn't leave her till he'd seen her put on the way back to her friends." "With a flea in her lug," suggested the farmer. "Your people never go to a clergyman in their distress. It's the last thing they'd think of. Any one might more probably be regarded as a friend than the parson. But with us the poor know where to look for sympathy." "She ain't that poor, neither," said the grandfather. "She had money with her?" "I don't know just what she had; but she ain't been brought up poor. And I don't think as our Ruby'd go of herself to any clergyman. It never was her way." "It never is the way with a Protestant," said the priest. "We'll say no more about that for the present," said Roger, who was waxing wroth with the priest. That a man should be fond of his own religion is right; but Roger Carbury was beginning to think that Father Barham was too fond of his religion. "What had we better do? I suppose we shall hear something of her at the railway. There are not so many people leaving Beccles but that she may be remembered." So the waggonette was ordered, and they all prepared to go off to the station together. But before they started John Crumb rode up to the door. He had gone at once to the farm on hearing of Ruby's departure, and had followed the farmer from thence to Carbury. Now he found the squire and the priest and the old man standing around as the horses were being put to the carriage. "Ye ain't a' found her, Mr. Ruggles, ha' ye?" he asked as he wiped the sweat from his brow. "Noa;—we ain't a' found no one yet." "If it was as she was to come to harm, Mr. Carbury, I'd never forgive myself,—never," said Crumb. "As far as I can understand it is no doing of yours, my friend," said the squire. "In one way, it ain't; and in one way it is. I was over there last night a bothering of her. She'd a' come round may be, if she'd a' been left alone. She wouldn't a' been off now, only for our going over to Sheep's Acre. But,—oh!" "What is it, Mr. Crumb?" "He's a coosin o' yours, squoire; and long as I've known Suffolk, I've never known nothing but good o' you and yourn. But if your baronite has been and done this! Oh, Mr. Carbury! If I was to wring his neck round, you wouldn't say as how I was wrong; would ye, now?" Roger could hardly answer the question. On general grounds the wringing of Sir Felix's neck, let the immediate cause for such a performance have been what it might, would have seemed to him to be a good deed. The world would be better, according to his thinking, with Sir Felix out of it than in it. But still the young man was his cousin and a Carbury, and to such a one as John Crumb he was bound to defend any member of his family as far as he might be defensible. "They says as how he was groping about Sheep's Acre when he was last here, a hiding himself and skulking behind hedges. Drat 'em all. They've gals enough of their own,—them fellows. Why can't they let a fellow alone? I'll do him a mischief, Master Roger; I wull;—if he's had a hand in this." Poor John Crumb! When he had his mistress to win he could find no words for himself; but was obliged to take an eloquent baker with him to talk for him. Now in his anger he could talk freely enough. "But you must first learn that Sir Felix has had anything to do with this, Mr. Crumb." "In coorse; in coorse. That's right. That's right. Must l'arn as he did it, afore I does it. But when I have l'arned!"— And John Crumb clenched his fist as though a very short lesson would suffice for him upon this occasion. They all went to the Beccles Station, and from thence to the Beccles post office,—so that Beccles soon knew as much about it as Bungay. At the railway station Ruby was distinctly remembered. She had taken a second-class ticket by the morning train for London, and had gone off without any appearance of secrecy. She had been decently dressed, with a hat and cloak, and her luggage had been such as she might have been expected to carry, had all her friends known that she was going. So much was made clear at the railway station, but nothing more could be learned there. Then a message was sent by telegraph to the station in London, and they all waited, loitering about the post office, for a reply. One of the porters in London remembered seeing such a girl as was described, but the man who was supposed to have carried her box for her to a cab had gone away for the day. It was believed that she had left the station in a four-wheel cab. "I'll be arter her. I'll be arter her at once," said John Crumb. But there was no train till night, and Roger Carbury was doubtful whether his going would do any good. It was evidently fixed on Crumb's mind that the first step towards finding Ruby would be the breaking of every bone in the body of Sir Felix Carbury. Now it was not at all apparent to the squire that his cousin had had anything to do with this affair. It had been made quite clear to him that the old man had quarrelled with his granddaughter and had threatened to turn her out of his house, not because she had misbehaved with Sir Felix, but on account of her refusing to marry John Crumb. John Crumb had gone over to the farm expecting to arrange it all, and up to that time there had been no fear about Felix Carbury. Nor was it possible that there should have been communication between Ruby and Felix since the quarrel at the farm. Even if the old man were right in supposing that Ruby and the baronet had been acquainted,—and such acquaintance could not but be prejudicial to the girl,—not on that account would the baronet be responsible for her abduction. John Crumb was thirsting for blood and was not very capable in his present mood of arguing the matter out coolly, and Roger, little as he loved his cousin, was not desirous that all Suffolk should know that Sir Felix Carbury had been thrashed within an inch of his life by John Crumb of Bungay. "I'll tell you what I'll do," said he, putting his hand kindly on the old man's shoulder. "I'll go up myself by the first train to-morrow. I can trace her better than Mr. Crumb can do, and you will both trust me." "There's not one in the two counties I'd trust so soon," said the old man. "But you'll let us know the very truth," said John Crumb. Roger Carbury made him an indiscreet promise that he would let him know the truth. So the matter was settled, and the grandfather and lover returned together to Bungay. About HackerNoon Book Series: We bring you the most important technical, scientific, and insightful public domain books. This book is part of the public domain. Anthony Trollope (2004). The Way We Live Now. Urbana, Illinois: Project Gutenberg. Retrieved October 2022 https://www.gutenberg.org/cache/epub/5231/pg5231-images.html This eBook is for the use of anyone anywhere at no cost and with almost no restrictions whatsoever. You may copy it, give it away or re-use it under the terms of the Project Gutenberg License included with this eBook or online at , located at . www.gutenberg.org https://www.gutenberg.org/policy/license.html

RUBY RUGGLES OBEYS HER GRANDFATHER

Delivering Cloud Infrastructure the GitOps Way with Crossplane and ArgoCD

What are Internal Developer Platforms (IDPs)? Internal developer platforms (IDPs) are platforms that developers can leverage to build and deploy their applications to one of the environments. Their main purpose is to increase the pace of development and reduce the developer’s dependence on DevOps and Platform engineers through automation and developer self-service. The platform may be built in-house, usually, an adaptation of an open-source or purchased offering such as platform as a Service (PaaS). The adoption has been both recent and rapid. Puppet’s state of platform engineering finds that over 51% of companies that have adopted Internal Developer Platforms have done it in the past three years. And an overwhelming majority of the respondents (93%) declared that IDP adoption is a step in the right direction. survey Internal developer platforms are good at helping the team manage applications and infrastructure from one place, providing tight integration with the existing tools and services, and giving developers self-serve and collaboration capabilities. Benefits of IDPs IDPs offer several benefits to the organizations using them. The first relates to improvements in infrastructure and IT management, such as increased productivity stemming from reduced communication times between devs and infra teams. Secondly, they reduce the complexity of the cloud and make it easy for people in your org to pick up on the best practices set forward by your team. In larger organizations, they can also be an easy way of managing RBAC to deployments, infra creation and management, and more. Responses from Puppet’s State of Platform Engineering 2023 report also show how IDP has improved : DevOps KPIs An increase in development velocity (68%) 42% say development velocity has improved “a great deal” since they started doing platform engineering Improved productivity (59%) Improved system reliability (60%) Improved security (55%) Components of IDPs Let’s dive deeper into the components of IDPs and discuss which parts of the application and infrastructure management process they improve. IDPs are built by integrating with the existing set of tools and services that the company currently uses. These can be your source control systems (e.g., GitHub, GitLab), Continuous Integration and Continuous Deployment (CI/CD) pipelines, and observability tools such as monitoring and logging systems. Integration with existing tools and services In addition, they may also include some form of access control, observability dashboards, and a set of best practices for developers to use. Application and infrastructure management IDPs can be used to automate a lot of the hassle of deployment. By setting up IDPs with GitOps best practices, users can automatically deploy application and infra changes with each commit to the Git code. Managing multiple environments is another popular use case of IDPs. They can be effective in creating, providing selective access, and managing the use of multiple different environments such as development, testing, pre-prod, and prod. Several tools set up preview environments on the fly. Their ability to monitor and view deployments across environments helps organizations scale dynamically to changing workloads and maintain consistent performance across their infrastructure. IDPs empower developers with self-serve capabilities by offering a centralized, user-friendly interface to access tools, resources, and services on demand. These capabilities facilitate faster development cycles, reduce operations teams’ dependency, and enable greater autonomy in seamlessly creating, testing, and deploying applications. Developer self-service capabilities The streamlined workflow also makes it considerably easier for new employees to get started with your stack without diving into the internal workings. Some tools also add a collaboration layer on top that allows different members of a team to work together easily and review changes before deploying the code. IDPs can achieve enhanced security and compliance by providing Role-based access controls (RBAC), ensuring each team member has the appropriate permissions to access specific resources and perform certain actions. Overall, this minimized the risk of unauthorized access or accidental changes while also promoting collaboration and efficient workflows among cross-functional teams. Collaboration and governance features IDPs help developer teams maintain transparency, accountability, and traceability throughout the development process by providing audit logs and history tracking. This enables easier identification and resolution of issues, as well as ensures compliance with regulatory and organizational requirements for data handling and change management. Platform engineering Companies that decide to build their own platform usually hire platform engineers, who are specialized professionals responsible for the creation and continuous improvement of the IDP. They work to implement and maintain tools, services, and best practices that streamline development processes, ensuring a smooth and efficient workflow for the organization's developers. Platform engineering is the process of designing, building, and maintaining an IDP that provides a centralized, scalable, and efficient infrastructure for developers within an organization. An IDP simplifies and standardizes application development, deployment, and management, enabling faster and more reliable software delivery. Software development trends that are catalysts for IDP adoption Digital transformation: As businesses worldwide increasingly embrace digital transformation, there is a growing need for efficient and robust software development processes. IDPs play a crucial role in enabling companies to stay agile and quickly adapt to the dynamic digital landscape. Need for faster software delivery: In today's fast-paced business environment, organizations must deliver new features and applications at an unprecedented speed. IDPs provide a standardized, automated, and centralized platform that accelerates development cycles, enabling companies to stay competitive and responsive to market changes. The growing complexity of software architectures: With the rise of microservices, containers, and cloud-native technologies, managing and deploying software has become increasingly complex. IDPs help simplify this complexity by providing a unified platform where developers can build, test, and deploy applications with ease, regardless of the underlying architecture. Demand for DevOps and CI/CD practices: The need for IDPs grows as more organizations adopt DevOps and Continuous Integration/Continuous Deployment (CI/CD) practices. IDPs enable seamless collaboration between development and operations teams, automating many manual tasks and ensuring smooth transitions throughout the software development lifecycle. Scalability and flexibility: IDPs offer a scalable solution that can accommodate the growing needs of organizations, regardless of their size. They provide a flexible platform that can be easily customized and adapted to cater to the unique requirements of different teams and projects. Cross-region collaboration: With businesses operating across multiple geographies, the need for a platform that supports cross-region collaboration is essential. IDPs empower development teams spread across the globe to work together seamlessly, enabling efficient knowledge sharing and fostering a culture of innovation. In summary, the rising popularity of IDPs across companies of all sizes and regions worldwide can be attributed to their ability to streamline software development processes, simplify complex architectures, support DevOps and CI/CD practices, and facilitate cross-region collaboration. As the demand for digitalization and agility continues to grow, IDPs are poised to play a critical role in shaping the future of software development. Commercially available internal developer platforms is a DevOps automation platform that helps engineering manage both the app and infra side of things and ship faster! Built with the GitOps best practices in mind, Argonaut reduces the complexities of creating and maintaining cloud setups. Whether it is Kubernetes app deployment to AWS or GCP, there are several runtimes, environments, regions, integrations, and app types to choose from. Argonaut offers a simple way to develop and operate modern cloud applications on Kubernetes. It can be adopted as either a self-hosted or PaaS option. There’s also a with essential plugins, templates, and applications making it easier to get started. Its features benefit not only the developers but also the platform engineers and CIOs. Mia Platform marketplace is an internal developer platform providing simplicity, automation, reusability, and self-service. It acts as a Platform Orchestrator that lets engineering teams remove bottlenecks by enabling them to build code-based golden paths (executable config files and templates) for developers. It can be used through the CLI or UI. Humanitec provides engineering teams self-serve access to tools and information. It helps developers ensure operational excellence across services with its that can be set up in a secure and compliant way. Opslevel integrations is a Kubernetes application management platform that enables efficient deployment processes. Developers can leverage its standardized application and policy definitions that are platform-agnostic. It also has a GUI-based portal to manage apps after deployment and gain visibility into pipelines for smoother operations. Shipa provides a context-rich software catalog with maturity and quality scorecards. It also supports comprehensive developer self-service actions while providing additional role-based access controls (RBAC). Their provides many key features and makes it a worthy contender. Port free-forever , powered by Crossplane, offers an enterprise-grade control plane solution for multi-cloud and hybrid environments, enabling efficient management of cloud infrastructure. The command line and make it more effective and easier to get started. Upbound Up Upbound marketplace is an all-in-one DevOps platform that streamlines the application deployment process. It provides a comprehensive feature set in a modular manner that allows teams to focus fully on business functionality. DevOpsBox Evaluating Internal Developer Platforms Despite its myriad of benefits, Internal Developer Platforms do not make sense for all teams. They can end up being overkill for certain types of engineering teams and a burden to build and maintain for companies with smaller teams. When they don’t make sense You have existing processes that are efficient. Don’t complicate things too early. Companies that are currently using PaaS or other managed offerings should continue doing so as long as possible. Limited resources and team size. This could also mean your team is small, and most of your team is senior and comfortable writing scripts and managing infrastructure. There is low development complexity. If you have just one app with a simple single-cloud setup. And, if your app is monolithic and doesn’t make use of the microservice architecture, there is little benefit from creating an IDP. Incompatible organizational culture. If an organization’s culture is resistant to change or does not foster collaboration and communication, implementing an IDP might not be successful and could even lead to decreased efficiency and productivity. When they make sense You plan to start using Microservices. This usually also means a growing development team size or the complexity of projects you handle. You have a small team, and not everyone feels comfortable with deployments, scripting, and infrastructure, and you have not yet hired a dedicated DevOps. Dependencies on other colleagues are blocking your developers. The cost of your existing setup (such as PaaS) is too high. Which is inevitable once you start to scale to meet new requirements. You have plans to go multi-cloud, adopt more modern cloud services, and scale geographically. You want to increase standardization and consistency across your teams. An IDP can help reduce errors, improve code quality, and ensure all developers work with the same set of tools and follow the same best practices. Also published here.

An In-Depth Guide to Internal Developer Platforms (IDPs)

Una guía detallada de las plataformas internas para desarrolladores (IDP)

Un guide détaillé des plates-formes de développement internes (IDP)

आंतरिक डेवलपर प्लेटफ़ॉर्म (IDPs) के लिए एक गहन मार्गदर्शिका

内部開発者プラットフォーム (IDP) の詳細ガイド

Um guia detalhado para plataformas internas de desenvolvedores (IDPs)

Hướng dẫn chuyên sâu về Nền tảng dành cho nhà phát triển nội bộ (IDP)

内部开发者平台 (IDP) 深度指南

The Strange Case Of Dr. Jekyll And Mr. Hyde by Robert Louis Stevenson is part of HackerNoon’s Book Blog Post series. You can jump to any chapter in this book here . Chapter IX: Dr. Lanyon's Narrative CHAPTER IX. DR. LANYON’S NARRATIVE On the ninth of January, now four days ago, I received by the evening delivery a registered envelope, addressed in the hand of my colleague and old school companion, Henry Jekyll. I was a good deal surprised by this; for we were by no means in the habit of correspondence; I had seen the man, dined with him, indeed, the night before; and I could imagine nothing in our intercourse that should justify formality of registration. The contents increased my wonder; for this is how the letter ran: “10th December, 18—. “Dear Lanyon,—You are one of my oldest friends; and although we may have differed at times on scientific questions, I cannot remember, at least on my side, any break in our affection. There was never a day when, if you had said to me, ‘Jekyll, my life, my honour, my reason, depend upon you,’ I would not have sacrificed my left hand to help you. Lanyon, my life, my honour, my reason, are all at your mercy; if you fail me to-night, I am lost. You might suppose, after this preface, that I am going to ask you for something dishonourable to grant. Judge for yourself. “I want you to postpone all other engagements for to-night—ay, even if you were summoned to the bedside of an emperor; to take a cab, unless your carriage should be actually at the door; and with this letter in your hand for consultation, to drive straight to my house. Poole, my butler, has his orders; you will find him waiting your arrival with a locksmith. The door of my cabinet is then to be forced; and you are to go in alone; to open the glazed press (letter E) on the left hand, breaking the lock if it be shut; and to draw out, with all its contents as they stand, the fourth drawer from the top or (which is the same thing) the third from the bottom. In my extreme distress of mind, I have a morbid fear of misdirecting you; but even if I am in error, you may know the right drawer by its contents: some powders, a phial and a paper book. This drawer I beg of you to carry back with you to Cavendish Square exactly as it stands. “That is the first part of the service: now for the second. You should be back, if you set out at once on the receipt of this, long before midnight; but I will leave you that amount of margin, not only in the fear of one of those obstacles that can neither be prevented nor foreseen, but because an hour when your servants are in bed is to be preferred for what will then remain to do. At midnight, then, I have to ask you to be alone in your consulting room, to admit with your own hand into the house a man who will present himself in my name, and to place in his hands the drawer that you will have brought with you from my cabinet. Then you will have played your part and earned my gratitude completely. Five minutes afterwards, if you insist upon an explanation, you will have understood that these arrangements are of capital importance; and that by the neglect of one of them, fantastic as they must appear, you might have charged your conscience with my death or the shipwreck of my reason. “Confident as I am that you will not trifle with this appeal, my heart sinks and my hand trembles at the bare thought of such a possibility. Think of me at this hour, in a strange place, labouring under a blackness of distress that no fancy can exaggerate, and yet well aware that, if you will but punctually serve me, my troubles will roll away like a story that is told. Serve me, my dear Lanyon and save “Your friend, “H.J. “P.S.—I had already sealed this up when a fresh terror struck upon my soul. It is possible that the post-office may fail me, and this letter not come into your hands until to-morrow morning. In that case, dear Lanyon, do my errand when it shall be most convenient for you in the course of the day; and once more expect my messenger at midnight. It may then already be too late; and if that night passes without event, you will know that you have seen the last of Henry Jekyll.” Upon the reading of this letter, I made sure my colleague was insane; but till that was proved beyond the possibility of doubt, I felt bound to do as he requested. The less I understood of this farrago, the less I was in a position to judge of its importance; and an appeal so worded could not be set aside without a grave responsibility. I rose accordingly from table, got into a hansom, and drove straight to Jekyll’s house. The butler was awaiting my arrival; he had received by the same post as mine a registered letter of instruction, and had sent at once for a locksmith and a carpenter. The tradesmen came while we were yet speaking; and we moved in a body to old Dr. Denman’s surgical theatre, from which (as you are doubtless aware) Jekyll’s private cabinet is most conveniently entered. The door was very strong, the lock excellent; the carpenter avowed he would have great trouble and have to do much damage, if force were to be used; and the locksmith was near despair. But this last was a handy fellow, and after two hour’s work, the door stood open. The press marked E was unlocked; and I took out the drawer, had it filled up with straw and tied in a sheet, and returned with it to Cavendish Square. Here I proceeded to examine its contents. The powders were neatly enough made up, but not with the nicety of the dispensing chemist; so that it was plain they were of Jekyll’s private manufacture; and when I opened one of the wrappers I found what seemed to me a simple crystalline salt of a white colour. The phial, to which I next turned my attention, might have been about half full of a blood-red liquor, which was highly pungent to the sense of smell and seemed to me to contain phosphorus and some volatile ether. At the other ingredients I could make no guess. The book was an ordinary version book and contained little but a series of dates. These covered a period of many years, but I observed that the entries ceased nearly a year ago and quite abruptly. Here and there a brief remark was appended to a date, usually no more than a single word: “double” occurring perhaps six times in a total of several hundred entries; and once very early in the list and followed by several marks of exclamation, “total failure!!!” All this, though it whetted my curiosity, told me little that was definite. Here were a phial of some salt, and the record of a series of experiments that had led (like too many of Jekyll’s investigations) to no end of practical usefulness. How could the presence of these articles in my house affect either the honour, the sanity, or the life of my flighty colleague? If his messenger could go to one place, why could he not go to another? And even granting some impediment, why was this gentleman to be received by me in secret? The more I reflected the more convinced I grew that I was dealing with a case of cerebral disease; and though I dismissed my servants to bed, I loaded an old revolver, that I might be found in some posture of self-defence. Twelve o’clock had scarce rung out over London, ere the knocker sounded very gently on the door. I went myself at the summons, and found a small man crouching against the pillars of the portico. “Are you come from Dr. Jekyll?” I asked. He told me “yes” by a constrained gesture; and when I had bidden him enter, he did not obey me without a searching backward glance into the darkness of the square. There was a policeman not far off, advancing with his bull’s eye open; and at the sight, I thought my visitor started and made greater haste. These particulars struck me, I confess, disagreeably; and as I followed him into the bright light of the consulting room, I kept my hand ready on my weapon. Here, at last, I had a chance of clearly seeing him. I had never set eyes on him before, so much was certain. He was small, as I have said; I was struck besides with the shocking expression of his face, with his remarkable combination of great muscular activity and great apparent debility of constitution, and—last but not least—with the odd, subjective disturbance caused by his neighbourhood. This bore some resemblance to incipient rigour, and was accompanied by a marked sinking of the pulse. At the time, I set it down to some idiosyncratic, personal distaste, and merely wondered at the acuteness of the symptoms; but I have since had reason to believe the cause to lie much deeper in the nature of man, and to turn on some nobler hinge than the principle of hatred. This person (who had thus, from the first moment of his entrance, struck in me what I can only describe as a disgustful curiosity) was dressed in a fashion that would have made an ordinary person laughable; his clothes, that is to say, although they were of rich and sober fabric, were enormously too large for him in every measurement—the trousers hanging on his legs and rolled up to keep them from the ground, the waist of the coat below his haunches, and the collar sprawling wide upon his shoulders. Strange to relate, this ludicrous accoutrement was far from moving me to laughter. Rather, as there was something abnormal and misbegotten in the very essence of the creature that now faced me—something seizing, surprising and revolting—this fresh disparity seemed but to fit in with and to reinforce it; so that to my interest in the man’s nature and character, there was added a curiosity as to his origin, his life, his fortune and status in the world. These observations, though they have taken so great a space to be set down in, were yet the work of a few seconds. My visitor was, indeed, on fire with sombre excitement. “Have you got it?” he cried. “Have you got it?” And so lively was his impatience that he even laid his hand upon my arm and sought to shake me. I put him back, conscious at his touch of a certain icy pang along my blood. “Come, sir,” said I. “You forget that I have not yet the pleasure of your acquaintance. Be seated, if you please.” And I showed him an example, and sat down myself in my customary seat and with as fair an imitation of my ordinary manner to a patient, as the lateness of the hour, the nature of my preoccupations, and the horror I had of my visitor, would suffer me to muster. “I beg your pardon, Dr. Lanyon,” he replied civilly enough. “What you say is very well founded; and my impatience has shown its heels to my politeness. I come here at the instance of your colleague, Dr. Henry Jekyll, on a piece of business of some moment; and I understood...” He paused and put his hand to his throat, and I could see, in spite of his collected manner, that he was wrestling against the approaches of the hysteria—“I understood, a drawer...” But here I took pity on my visitor’s suspense, and some perhaps on my own growing curiosity. “There it is, sir,” said I, pointing to the drawer, where it lay on the floor behind a table and still covered with the sheet. He sprang to it, and then paused, and laid his hand upon his heart; I could hear his teeth grate with the convulsive action of his jaws; and his face was so ghastly to see that I grew alarmed both for his life and reason. “Compose yourself,” said I. He turned a dreadful smile to me, and as if with the decision of despair, plucked away the sheet. At sight of the contents, he uttered one loud sob of such immense relief that I sat petrified. And the next moment, in a voice that was already fairly well under control, “Have you a graduated glass?” he asked. I rose from my place with something of an effort and gave him what he asked. He thanked me with a smiling nod, measured out a few minims of the red tincture and added one of the powders. The mixture, which was at first of a reddish hue, began, in proportion as the crystals melted, to brighten in colour, to effervesce audibly, and to throw off small fumes of vapour. Suddenly and at the same moment, the ebullition ceased and the compound changed to a dark purple, which faded again more slowly to a watery green. My visitor, who had watched these metamorphoses with a keen eye, smiled, set down the glass upon the table, and then turned and looked upon me with an air of scrutiny. “And now,” said he, “to settle what remains. Will you be wise? will you be guided? will you suffer me to take this glass in my hand and to go forth from your house without further parley? or has the greed of curiosity too much command of you? Think before you answer, for it shall be done as you decide. As you decide, you shall be left as you were before, and neither richer nor wiser, unless the sense of service rendered to a man in mortal distress may be counted as a kind of riches of the soul. Or, if you shall so prefer to choose, a new province of knowledge and new avenues to fame and power shall be laid open to you, here, in this room, upon the instant; and your sight shall be blasted by a prodigy to stagger the unbelief of Satan.” “Sir,” said I, affecting a coolness that I was far from truly possessing, “you speak enigmas, and you will perhaps not wonder that I hear you with no very strong impression of belief. But I have gone too far in the way of inexplicable services to pause before I see the end.” “It is well,” replied my visitor. “Lanyon, you remember your vows: what follows is under the seal of our profession. And now, you who have so long been bound to the most narrow and material views, you who have denied the virtue of transcendental medicine, you who have derided your superiors—behold!” He put the glass to his lips and drank at one gulp. A cry followed; he reeled, staggered, clutched at the table and held on, staring with injected eyes, gasping with open mouth; and as I looked there came, I thought, a change—he seemed to swell—his face became suddenly black and the features seemed to melt and alter—and the next moment, I had sprung to my feet and leaped back against the wall, my arms raised to shield me from that prodigy, my mind submerged in terror. “O God!” I screamed, and “O God!” again and again; for there before my eyes—pale and shaken, and half fainting, and groping before him with his hands, like a man restored from death—there stood Henry Jekyll! What he told me in the next hour, I cannot bring my mind to set on paper. I saw what I saw, I heard what I heard, and my soul sickened at it; and yet now when that sight has faded from my eyes, I ask myself if I believe it, and I cannot answer. My life is shaken to its roots; sleep has left me; the deadliest terror sits by me at all hours of the day and night; and I feel that my days are numbered, and that I must die; and yet I shall die incredulous. As for the moral turpitude that man unveiled to me, even with tears of penitence, I cannot, even in memory, dwell on it without a start of horror. I will say but one thing, Utterson, and that (if you can bring your mind to credit it) will be more than enough. The creature who crept into my house that night was, on Jekyll’s own confession, known by the name of Hyde and hunted for in every corner of the land as the murderer of Carew. HASTIE LANYON. About HackerNoon Book Series: We bring you the most important technical, scientific, and insightful public domain books. This book is part of the public domain. Stevenson, Robert Louis, 1992. The Strange Case Of Dr. Jekyll And Mr. Hyde. Urbana, Illinois: Project Gutenberg. Retrieved May 2022 from https://www.gutenberg.org/files/43/43-h/43-h.htm#chap09 This eBook is for the use of anyone anywhere at no cost and with almost no restrictions whatsoever. You may copy it, give it away or re-use it under the terms of the Project Gutenberg License included with this eBook or online at www.gutenberg.org, located at https://www.gutenberg.org/policy/license.html.

The Strange Case Of Dr. Jekyll And Mr. Hyde: Chapter IX - Dr. Lanyon's Narrative 

Photo by on Jason Blackeye Unsplash Adventures of Huckleberry Finn, by Mark Twain (Samuel Clemens) is part of HackerNoon’s Book Blog Post series. You can jump to any chapter in this book . here CHAPTER XII. It must a been close on to one o’clock when we got below the island at last, and the raft did seem to go mighty slow. If a boat was to come along we was going to take to the canoe and break for the Illinois shore; and it was well a boat didn’t come, for we hadn’t ever thought to put the gun in the canoe, or a fishing-line, or anything to eat. We was in ruther too much of a sweat to think of so many things. It warn’t good judgment to put everything on the raft. If the men went to the island I just expect they found the camp fire I built, and watched it all night for Jim to come. Anyways, they stayed away from us, and if my building the fire never fooled them it warn’t no fault of mine. I played it as low down on them as I could. When the first streak of day began to show we tied up to a towhead in a big bend on the Illinois side, and hacked off cottonwood branches with the hatchet, and covered up the raft with them so she looked like there had been a cave-in in the bank there. A tow-head is a sandbar that has cottonwoods on it as thick as harrow-teeth. We had mountains on the Missouri shore and heavy timber on the Illinois side, and the channel was down the Missouri shore at that place, so we warn’t afraid of anybody running across us. We laid there all day, and watched the rafts and steamboats spin down the Missouri shore, and up-bound steamboats fight the big river in the middle. I told Jim all about the time I had jabbering with that woman; and Jim said she was a smart one, and if she was to start after us herself she wouldn’t set down and watch a camp fire—no, sir, she’d fetch a dog. Well, then, I said, why couldn’t she tell her husband to fetch a dog? Jim said he bet she did think of it by the time the men was ready to start, and he believed they must a gone up-town to get a dog and so they lost all that time, or else we wouldn’t be here on a towhead sixteen or seventeen mile below the village—no, indeedy, we would be in that same old town again. So I said I didn’t care what was the reason they didn’t get us as long as they didn’t. When it was beginning to come on dark we poked our heads out of the cottonwood thicket, and looked up and down and across; nothing in sight; so Jim took up some of the top planks of the raft and built a snug wigwam to get under in blazing weather and rainy, and to keep the things dry. Jim made a floor for the wigwam, and raised it a foot or more above the level of the raft, so now the blankets and all the traps was out of reach of steamboat waves. Right in the middle of the wigwam we made a layer of dirt about five or six inches deep with a frame around it for to hold it to its place; this was to build a fire on in sloppy weather or chilly; the wigwam would keep it from being seen. We made an extra steering-oar, too, because one of the others might get broke on a snag or something. We fixed up a short forked stick to hang the old lantern on, because we must always light the lantern whenever we see a steamboat coming down-stream, to keep from getting run over; but we wouldn’t have to light it for up-stream boats unless we see we was in what they call a “crossing”; for the river was pretty high yet, very low banks being still a little under water; so up-bound boats didn’t always run the channel, but hunted easy water. This second night we run between seven and eight hours, with a current that was making over four mile an hour. We catched fish and talked, and we took a swim now and then to keep off sleepiness. It was kind of solemn, drifting down the big, still river, laying on our backs looking up at the stars, and we didn’t ever feel like talking loud, and it warn’t often that we laughed—only a little kind of a low chuckle. We had mighty good weather as a general thing, and nothing ever happened to us at all—that night, nor the next, nor the next. Every night we passed towns, some of them away up on black hillsides, nothing but just a shiny bed of lights; not a house could you see. The fifth night we passed St. Louis, and it was like the whole world lit up. In St. Petersburg they used to say there was twenty or thirty thousand people in St. Louis, but I never believed it till I see that wonderful spread of lights at two o’clock that still night. There warn’t a sound there; everybody was asleep. Every night now I used to slip ashore towards ten o’clock at some little village, and buy ten or fifteen cents’ worth of meal or bacon or other stuff to eat; and sometimes I lifted a chicken that warn’t roosting comfortable, and took him along. Pap always said, take a chicken when you get a chance, because if you don’t want him yourself you can easy find somebody that does, and a good deed ain’t ever forgot. I never see pap when he didn’t want the chicken himself, but that is what he used to say, anyway. Mornings before daylight I slipped into cornfields and borrowed a watermelon, or a mushmelon, or a punkin, or some new corn, or things of that kind. Pap always said it warn’t no harm to borrow things if you was meaning to pay them back some time; but the widow said it warn’t anything but a soft name for stealing, and no decent body would do it. Jim said he reckoned the widow was partly right and pap was partly right; so the best way would be for us to pick out two or three things from the list and say we wouldn’t borrow them any more—then he reckoned it wouldn’t be no harm to borrow the others. So we talked it over all one night, drifting along down the river, trying to make up our minds whether to drop the watermelons, or the cantelopes, or the mushmelons, or what. But towards daylight we got it all settled satisfactory, and concluded to drop crabapples and p’simmons. We warn’t feeling just right before that, but it was all comfortable now. I was glad the way it come out, too, because crabapples ain’t ever good, and the p’simmons wouldn’t be ripe for two or three months yet. We shot a water-fowl now and then that got up too early in the morning or didn’t go to bed early enough in the evening. Take it all round, we lived pretty high. The fifth night below St. Louis we had a big storm after midnight, with a power of thunder and lightning, and the rain poured down in a solid sheet. We stayed in the wigwam and let the raft take care of itself. When the lightning glared out we could see a big straight river ahead, and high, rocky bluffs on both sides. By and by says I, “Hel-lo, Jim, looky yonder!” It was a steamboat that had killed herself on a rock. We was drifting straight down for her. The lightning showed her very distinct. She was leaning over, with part of her upper deck above water, and you could see every little chimbly-guy clean and clear, and a chair by the big bell, with an old slouch hat hanging on the back of it, when the flashes come. Well, it being away in the night and stormy, and all so mysterious-like, I felt just the way any other boy would a felt when I see that wreck laying there so mournful and lonesome in the middle of the river. I wanted to get aboard of her and slink around a little, and see what there was there. So I says: “Le’s land on her, Jim.” But Jim was dead against it at first. He says: “I doan’ want to go fool’n ’long er no wrack. We’s doin’ blame’ well, en we better let blame’ well alone, as de good book says. Like as not dey’s a watchman on dat wrack.” “Watchman your grandmother,” I says; “there ain’t nothing to watch but the texas and the pilot-house; and do you reckon anybody’s going to resk his life for a texas and a pilot-house such a night as this, when it’s likely to break up and wash off down the river any minute?” Jim couldn’t say nothing to that, so he didn’t try. “And besides,” I says, “we might borrow something worth having out of the captain’s stateroom. Seegars, I bet you—and cost five cents apiece, solid cash. Steamboat captains is always rich, and get sixty dollars a month, and they don’t care a cent what a thing costs, you know, long as they want it. Stick a candle in your pocket; I can’t rest, Jim, till we give her a rummaging. Do you reckon Tom Sawyer would ever go by this thing? Not for pie, he wouldn’t. He’d call it an adventure—that’s what he’d call it; and he’d land on that wreck if it was his last act. And wouldn’t he throw style into it?—wouldn’t he spread himself, nor nothing? Why, you’d think it was Christopher C’lumbus discovering Kingdom-Come. I wish Tom Sawyer was here.” Jim he grumbled a little, but give in. He said we mustn’t talk any more than we could help, and then talk mighty low. The lightning showed us the wreck again just in time, and we fetched the stabboard derrick, and made fast there. The deck was high out here. We went sneaking down the slope of it to labboard, in the dark, towards the texas, feeling our way slow with our feet, and spreading our hands out to fend off the guys, for it was so dark we couldn’t see no sign of them. Pretty soon we struck the forward end of the skylight, and clumb on to it; and the next step fetched us in front of the captain’s door, which was open, and by Jimminy, away down through the texas-hall we see a light! and all in the same second we seem to hear low voices in yonder! Jim whispered and said he was feeling powerful sick, and told me to come along. I says, all right, and was going to start for the raft; but just then I heard a voice wail out and say: “Oh, please don’t, boys; I swear I won’t ever tell!” Another voice said, pretty loud: “It’s a lie, Jim Turner. You’ve acted this way before. You always want more’n your share of the truck, and you’ve always got it, too, because you’ve swore ’t if you didn’t you’d tell. But this time you’ve said it jest one time too many. You’re the meanest, treacherousest hound in this country.” By this time Jim was gone for the raft. I was just a-biling with curiosity; and I says to myself, Tom Sawyer wouldn’t back out now, and so I won’t either; I’m a-going to see what’s going on here. So I dropped on my hands and knees in the little passage, and crept aft in the dark till there warn’t but one stateroom betwixt me and the cross-hall of the texas. Then in there I see a man stretched on the floor and tied hand and foot, and two men standing over him, and one of them had a dim lantern in his hand, and the other one had a pistol. This one kept pointing the pistol at the man’s head on the floor, and saying: “I’d like to! And I orter, too—a mean skunk!” The man on the floor would shrivel up and say, “Oh, please don’t, Bill; I hain’t ever goin’ to tell.” And every time he said that the man with the lantern would laugh and say: “’Deed you ain’t! You never said no truer thing ’n that, you bet you.” And once he said: “Hear him beg! and yit if we hadn’t got the best of him and tied him he’d a killed us both. And what for? Jist for noth’n. Jist because we stood on our rights—that’s what for. But I lay you ain’t a-goin’ to threaten nobody any more, Jim Turner. Put up that pistol, Bill.” Bill says: “I don’t want to, Jake Packard. I’m for killin’ him—and didn’t he kill old Hatfield jist the same way—and don’t he deserve it?” “But I don’t want him killed, and I’ve got my reasons for it.” “Bless yo’ heart for them words, Jake Packard! I’ll never forgit you long’s I live!” says the man on the floor, sort of blubbering. Packard didn’t take no notice of that, but hung up his lantern on a nail and started towards where I was there in the dark, and motioned Bill to come. I crawfished as fast as I could about two yards, but the boat slanted so that I couldn’t make very good time; so to keep from getting run over and catched I crawled into a stateroom on the upper side. The man came a-pawing along in the dark, and when Packard got to my stateroom, he says: “Here—come in here.” And in he come, and Bill after him. But before they got in I was up in the upper berth, cornered, and sorry I come. Then they stood there, with their hands on the ledge of the berth, and talked. I couldn’t see them, but I could tell where they was by the whisky they’d been having. I was glad I didn’t drink whisky; but it wouldn’t made much difference anyway, because most of the time they couldn’t a treed me because I didn’t breathe. I was too scared. And, besides, a body couldn’t breathe and hear such talk. They talked low and earnest. Bill wanted to kill Turner. He says: “He’s said he’ll tell, and he will. If we was to give both our shares to him now it wouldn’t make no difference after the row and the way we’ve served him. Shore’s you’re born, he’ll turn State’s evidence; now you hear me. I’m for putting him out of his troubles.” “So’m I,” says Packard, very quiet. “Blame it, I’d sorter begun to think you wasn’t. Well, then, that’s all right. Le’s go and do it.” “Hold on a minute; I hain’t had my say yit. You listen to me. Shooting’s good, but there’s quieter ways if the thing’s got to be done. But what I say is this: it ain’t good sense to go court’n around after a halter if you can git at what you’re up to in some way that’s jist as good and at the same time don’t bring you into no resks. Ain’t that so?” “You bet it is. But how you goin’ to manage it this time?” “Well, my idea is this: we’ll rustle around and gather up whatever pickins we’ve overlooked in the staterooms, and shove for shore and hide the truck. Then we’ll wait. Now I say it ain’t a-goin’ to be more’n two hours befo’ this wrack breaks up and washes off down the river. See? He’ll be drownded, and won’t have nobody to blame for it but his own self. I reckon that’s a considerble sight better ’n killin’ of him. I’m unfavorable to killin’ a man as long as you can git aroun’ it; it ain’t good sense, it ain’t good morals. Ain’t I right?” “Yes, I reck’n you are. But s’pose she don’t break up and wash off?” “Well, we can wait the two hours anyway and see, can’t we?” “All right, then; come along.” So they started, and I lit out, all in a cold sweat, and scrambled forward. It was dark as pitch there; but I said, in a kind of a coarse whisper, “Jim!” and he answered up, right at my elbow, with a sort of a moan, and I says: “Quick, Jim, it ain’t no time for fooling around and moaning; there’s a gang of murderers in yonder, and if we don’t hunt up their boat and set her drifting down the river so these fellows can’t get away from the wreck there’s one of ’em going to be in a bad fix. But if we find their boat we can put all of ’em in a bad fix—for the sheriff ’ll get ’em. Quick—hurry! I’ll hunt the labboard side, you hunt the stabboard. You start at the raft, and—” “Oh, my lordy, lordy! raf’’? Dey ain’ no raf’ no mo’; she done broke loose en gone I—en here we is!” About HackerNoon Book Series: We bring you the most important technical, scientific, and insightful public domain books. This book is part of the public domain. Urbana, Illinois: Project Gutenberg. Retrieved April 2022 from Twain, Mark,1993. Adventures of Huckleberry Finn. https://www.gutenberg.org/files/76/76-h/76-h.htm#c12 This eBook is for the use of anyone anywhere at no cost and with almost no restrictions whatsoever. You may copy it, give it away or re-use it under the terms of the Project Gutenberg License included with this eBook or online at , located at . www.gutenberg.org https://www.gutenberg.org/policy/license.html

Adventures of Huckleberry Finn: Chapter XII

Introduction In this blog, we will look at the evolution of software infrastructure; provisioning, delivery and maintenance. If you are interested in modern DevOps and SRE practices, this article is for you. is a common pattern where virtualized infrastructure and auxiliary services can be managed using configuration expressed in almost any language, usually hosted in a source code repository. Infrastructure as Code (IaC) Every once in a while software industry is shaped by significant events called . Paradigm Shift Here are a few such events that made Infrastructure as Code what it is today: Virtualization Containerisation and containers orchestration Public Cloud Infrastructure DevOps Culture Infrastructure provisioning: evolution Let’s examine how each wave of changes shaped the software landscape: #1 Virtualization In the early days of virtualization, developers would produce software in long waterfall development cycles, mostly based on requirements documentation. Ops Teams at the same time would rack and stack servers, provision infrastructure components, install all the things, configure etc. Typically both teams would work in a disconnected fashion, opening tickets and communicating via long email chains. Those were the darg ages. #2 Agile and beginning of DevOps Next came the agile revolution and with it the beginning of the DevOps culture. Now developers would send the application over to the Ops Team along with a multi-page configuration manual. More progressive teams will start collaborating on automation tools. In the early days of infrastructure automation tools like Chef or Puppet were very popular. Huge improvement, but often very siloed environment and fragmented responsibilities. #3 Public Cloud & container orchestration Along with increased usage of the public cloud and more robust automation, things started to fall into the right places. The tool of choice for most of the OPS Teams was terraform. Compared to the situation from a few years ago, this is huge progress. The amount of infrastructure created grows exponentially. For a long time, this picture is almost perfect… almost. So what is missing? Well, the experience for both Dev and Ops falls a bit short. Meticulously created infrastructure starts to drift, requires a lot of effort and maintenance overhead. Developers on the other hand must learn new languages and struggle to incorporate the Ops tooling into their workflows. Community rallies around and comes up with all kinds of creative solutions, drift detection, more automation, observability etc. All those tools and projects are great, but not standardised. How to improve the already good process? How to make one more leap to make something good, great? The answer came, like most good answers, from within the community itself. Remember how we talked about paradigm shifts? One of them was containerization and containers orchestration, this enabled standardization of to lesser extent packaging (container images) and runtime (Kubernetes pods) concerns. Now we have all the standards and tools at our fingertips, just one final touch needed to make it work seamlessly together. Here is how, in my opinion, is the missing link in the infrastructure as code evolution. Crossplane The power of Crossplane comes from using cloud native open standards and most popular tools to enable Developers (aka Application Teams) and Ops (aka Platform Teams) to work together, but without depending on each other. Before we move on to the next part of the blog, a little disclaimer. Choosing cows and penguins to represent developers and ops respectively, was based purely on the cuteness level of the animal. Crossplane Architecture What makes Crossplane so special? First, it builds on Kubernetes and capitalizes on the fact that the real power of Kubernetes is its powerful API model and control plane logic (control loops). It also moves away from Infrastructure as Code to Infrastructure as Data. The difference is that IaC means writing code to describe how the provisioning should happen, whereas IaD means writing pure data files (in the case of Kubernetes YAML) and submitting them to the control component (in the case of Kubernetes an operator) to encapsulate and execute the provisioning logic. For me, the best part about Crossplane is that it seamlessly enables collaboration between Application Teams and Platform Teams, by leveraging Plane as the convergence point where everyone meets. Kubernetes Control Components Let’s look how the Crossplane component model looks like. A word of warning, if you are new to Kubernetes this might be overwhelming, but there is value in making an effort to understand it. The below diagram shows the Crossplane component model and its basic interactions. Demo Scenario The demo scenario highlights Crossplane’s composite functionality. Using composites helps abstract away infrastructure complexity from developers by moving it into a Platform Team. The scenario flow: deploy RDS deploy EKS highlight the main steps of creating and using a composition Prerequisites To follow along, you will need a subscription to AWS and CLI configured on your local machine. Credentials for accessing cloud environments and deploying the infrastructure will be mapped from mounted volumes. Locally installed you will need: VS Code with remote containers devcontainer plugin WSL2 if using Windows Demo Setup All components from the are already pre-installed in the devcontainer in this project. Crossplane installation The components are: Kubernetes minikube Helm kubectl Crossplane CLI AWS CLI Crossplane should be installed in a namespace, if not please run crossplane-system .devcontainer/library-scripts/setup-crossplane.sh This setup is a little bit clunky, for example, performance is sub-optimal due to files being read from the container and the overall setup is not where I want it to be. This might change into a vagrant based setup if the containers reach the limit of abuse. Observability To visualize CRDs use ; a VMWare open-source cluster visualizer, running in a browser so no in-cluster installation is required. Octant If you like terminal tools more, got you covered. k9s AWS provider The setup will automatically generate AWS configuration based on the default profile mounted from your local $HOME/.aws folder. creds.conf file is added to .gitignore so you will not commit it to the repo accidentally! The next step is to configure Crossplane to access AWS and create resources, we will achieve this by creating a secret: kubectl create secret generic aws-creds -n crossplane-system --from-file=creds=./creds.conf and now, install AWS provider on the cluster kubectl apply -f https://raw.githubusercontent.com/crossplane/crossplane/release-1.5/docs/snippets/configure/aws/providerconfig.yaml From there onwards you should be able to follow along with the demos from . Crossplane's web page Deploy RDS Instance First let's deploy an RDS Instance, which is an AWS managed resource and comes with the AWS provider. See . Crossplane components diagram kubectl create -f rds-instance.yaml RDSInstance is just a Kubernetes resource like pod, service or replicaSet. You can check the deployment progress in Octant or command line: watch kubectl get RDSInstance Delete the instance with kubectl delete -f rds-instance.yaml Deploy EKS Cluster Deploying EKS cluster with automation is not a trivial task, there are a lot of components that need to be created along, like VPCs, subnets, IAM Roles, node pools, route tables, gateways... This is not the experience developers want, but this complexity must go somewhere, there is no magic "Remove complexity" button! As we've seen in the component diagram, the process starts with building an XRD (Composite Resource Definition) where we can specify: schema of the XR (Composite Resource) schema of the XRC (Composite Resource Claim) To help visualize how complex the configuration can be, here is a diagram representing the EKS Composition. This complexity is managed by Platform Team, folks that specialize in both Kubernetes and Cloud Providers. Now with this complexity hidden, developers are getting this: apiVersion: crossplanedemo.com/v1alpha1
kind: CompositeKubernetesCluster
metadata:
  name: devops-team
  labels:
    cluster-owner: piotrzan
spec:
  compositionRef:
    # Possible values cluster-google, cluster-azure, cluster-aws
    name: cluster-aws
  parameters:
    # Possible values small, medium, large
    nodeSize: small
    # version: "1.20"
    # minNodeCount: 2
  writeConnectionSecretToRef:
    namespace: devops-team
    name: cluster Deploying cluster is very easy, just kubectl create -f ./aws EKS Cluster deployment status: Retrieve kubeconfig details kubectl get secrets --namespace devops-team cluster \
     --output jsonpath="{.data.kubeconfig}" \
     | base64 --decode | tee eks-config.yaml

export KUBECONFIG=$PWD/eks-config.yaml From here you can use the cluster like your normal EKS. Remember to or source bash/zshrc to get your old config back. unset KUBECONFIG Cleanup Delete EKS and accompanying resources Exit VS Code which will stop the container. You can further clean unused containers and images by . kubectl delete -f ./aws docker system prune --all Conclusion Just in case anyone is wondering, this article is not sponsored by Upbound (the company behind Crossplane), I genuinely believe that this product is revolutionary. We have seen how Crossplane can help make infrastructure provisioning and management easier. Here are a few benefits I would like to highlight. Composable Infrastructure Self-Service Increased Automation Standardized collaboration Ubiquitous language (K8s API) What I like most about Crossplane is that it’s built with the DevOps culture in mind by promoting loosely coupled collaboration between Applications Teams and Platform Teams. The resource model, packaging, configuration are well thought out. There are also a few challenges to keep in mind: Complexity, it’s a price to pay for the flexibility it provides YAML proliferation, which is good or bad depending on where you stand on YAML ;) You need to know K8s well Complexity is addressed by moving it to specialized Platform Teams. For YAML I would love to see more push for integrating YAML generation like CDK8s or others. I see reliance on K8s as a benefit, but for those of us who are not yet comfortable with Kubernetes, this makes the learning curve a bit steeper. In summary, Crossplane is a great product, it appeared at the right time and solves decades all problems in a very innovative and future proof way. Go check it out!

Infrastructure As Code: The Next Big Shift is Here

Linear programming is one of the most common optimization techniques. It has a wide range of applications and is frequently used in operations research, industrial design, planning, and the list goes on. Alas, it is not as hyped as machine learning is (which is certainly a form of optimization itself), but is the go-to method for problems that can be formulated through decision variables that have linear relationships. This is a fast practical tutorial, I will perhaps cover the Simplex algorithm and the theory in a later post. Problem Statement Just to get an idea, we are going to solve a simple problem regarding production scheduling. Imagine that you work for a company that builds computers. A computer is a fairly complex product, and there are several factories that assemble them which the company pays a certain amount per unit. The cost of this computer model on the market is fixed at 500$, different factories assemble the computers at different speeds and costs. Factory produces 2000 per day at 450$ per unit, factory 1500 per day at 420$ per unit and 1000 per day at 400$ per unit. We have 1 month to assemble 80 000 units under the constraint that no factory is to produce more than double the units than any other factory. The question is, such that we obtained from selling the computers under those constraints? f0 f1 f2 what is the optimal production allocation between the factories maximize the profit In this tutorial we are going to be using Python and a linear programming optimization package , copy-paste install with pip: PuLP pip install pulp Now, in order to solve the computer production problem with linear programming, we need the following things: The set of decision variables The set of linear constraints on those variables A linear objective function to maximize or minimize So, open up your favorite editor and let's get started. Before defining the variables in PuLP, we need to create a problem object with the following code: from pulp import *problem = LpProblem("problemName", LpMaximize) We are going to add the constraints and objective function to this object. Notice that the problem constructor receives a problem name and , which means we want to maximize our objective function. In our case, this is the profit from selling a certain number of computers. Additionally, we also define the constants that we received from the problem statement: LpMaximize # factory cost per daycf0 = 450cf1 = 420cf2 = 400 # factory throughput per dayf0 = 2000f1 = 1500f2 = 1000 # production goalgoal = 80000 # time limitmax_num_days = 30 num_factories = 3 In the next section, we will define the decision variables. Decision Variables In PuLP, a decision variable is defined the following way: variable = LpVariable("variableName") Sometimes we need to provide bounds to the variable (the default is no bounds), in this case, we would write the following: var = LpVariable("boundedVariableName",lowerBound,upperBound) Yet another useful way to define variables in PuLP is using the dicts function. This can be very useful in cases where we need to define a large number of variables of the same type and bounds, would be a list of keys for the dictionary: variableNames varDict = LpVariable.dicts("varDict", variableNames, lowBound, upBound) So based on the previous definitions, the decision variables for the computer production problem are the days that we spend producing for each factory: # factoriesnum_factories = 3factory_days = LpVariable.dicts("factoryDays", list(range(num_factories)), 0, 30, cat="Continuous") Constraints Now that we defined our decision variables we can shift to defining the constraints for the problem. Note that the constraints have to be linear in a linear programming setting. The constraints that we care about are that the number of units assembled should be above or equal to the goal amount and the production constraint that no factory should produce more than double as the other factory: # goal constraintc1 = factory_days[0]*f1 + factory_days[1]*f2 + factory_days[2] * f3 >= goal # production constraintsc2 = factory_days[0]*f0 <= 2*factory_days[1]*f1c3 = factory_days[0]*f0 <= 2*factory_days[2]*f2c4 = factory_days[1]*f1 <= 2*factory_days[2]*f2c5 = factory_days[1]*f1 <= 2*factory_days[0]*f0c6 = factory_days[2]*f2 <= 2*factory_days[1]*f1c7 = factory_days[2]*f2 <= 2*factory_days[0]*f0 # adding the constraints to the problemproblem += c1problem += c2problem += c3problem += c4problem += c5problem += c6problem += c7 The objective function for the computer assembly problem is basically minimizing the cost of assembling all of those computers. This can be written simply as maximizing the negative cost: # objective functionproblem += -factory_days[0]*cf0*f0 - factory_days[1]*cf1*f1 - factory_days[2]*cf2*f2 Let's take a look at the problem definition, this can be simply achieved by the call: print(problem) This results in the following output which I think is self-explanatory, it lists the objective function, the constraints and the various decision variables in the problem: computerAssembly:MAXIMIZE-900000*factoryDays_0 + -630000*factoryDays_1 + -400000*factoryDays_2 + 0SUBJECT TO_C1: 1500 factoryDays_0 + 1000 factoryDays_1 + 1000 factoryDays_2 >= 80000 _C2: 2000 factoryDays_0 - 3000 factoryDays_1 <= 0 _C3: 2000 factoryDays_0 - 2000 factoryDays_2 <= 0 _C4: 1500 factoryDays_1 - 2000 factoryDays_2 <= 0 _C5: - 4000 factoryDays_0 + 1500 factoryDays_1 <= 0 _C6: - 3000 factoryDays_1 + 1000 factoryDays_2 <= 0 _C7: - 4000 factoryDays_0 + 1000 factoryDays_2 <= 0 VARIABLESfactoryDays_0 <= 30 ContinuousfactoryDays_1 <= 30 ContinuousfactoryDays_2 <= 30 Continuous Solving After we defined everything necessary in our linear programming problem, we can call the solve method which outputs 1 if the problem is solved and -1 if it was infeasible, it is a simple one-liner: # solvingproblem.solve() The solutions to the problem can be obtained by accessing the varValue attribute of each variable: for i in range(3):print(f"Factory {i}: {factory_days[i].varValue}") This results in the following output: Factory 0: 23.076923Factory 1: 15.384615Factory 2: 30.0 In linear programming, we assume that the relationships between the variables are linear and that the variables themselves are continuous. As a follow up on this tutorial, I will be covering Mixed Integer Programming, where the variables can be integers, which will prove a very useful thing since it can be used to simulate boolean logic.

Linear Programming in Python: A Straight Forward Tutorial

https://www.youtube.com/watch?v=XkiSUYxFaoY&embedable=true&transcript=true Here’s my new interview with the . Their new podcast (I’m the third guest!) is . I am also testing out HackerNoon’s new Youtube embed + transcript function.  Featured image credit is for prompt of ‘ I haven’t watched it yet, but I do recall having an interesting discussion about how a founder’s personality will make it’s way into the company’s operations one way or the other. SaaS Group SaaS Unbound HackerNoon AI mountain man takes zoom call in front of green pixelated clock. three percent of those watching go on to start uber successful saas companies.’

About Scaling a Startup and Growing a Dedicated Fully-Remote Software Team [SaaS Unbound Podcast] 

একটি স্টার্টআপ স্কেল করা এবং একটি ডেডিকেটেড সম্পূর্ণ-রিমোট সফ্টওয়্যার টিম বৃদ্ধি করা সম্পর্কে [সাস আনবাউন্ড পডকাস্ট]

Über die Skalierung eines Startups und den Aufbau eines dedizierten, vollständig remote arbeitenden Softwareteams [SaaS Unbound Podcast]

Acerca de escalar una startup y hacer crecer un equipo de software totalmente remoto dedicado [SaaS Unbound Podcast]

À propos de la mise à l'échelle d'une startup et de la croissance d'une équipe logicielle dédiée entièrement à distance [SaaS Unbound Podcast]

एक स्टार्टअप का विस्तार करने और समर्पित पूर्ण-रिमोट सॉफ्टवेयर टीम [सास अनबाउंड पॉडकास्ट] विकसित करने के बारे में

スタートアップのスケーリングと専用の完全リモート ソフトウェア チームの成長について [SaaS Unbound Podcast]

스타트업 확장 및 전담 완전 원격 소프트웨어 팀 성장 정보 [SaaS 언바운드 팟캐스트]

Sobre o dimensionamento de uma startup e o crescimento de uma equipe dedicada de software totalmente remota [SaaS Unbound Podcast]

Bir Startup'ı Ölçeklendirme ve Tamamen Uzaktan Özel Bir Yazılım Ekibini Büyütme Hakkında [SaaS Unbound Podcast]

Giới thiệu về Mở rộng Quy mô Khởi nghiệp và Phát triển Nhóm Phần mềm Từ xa Hoàn toàn Chuyên dụng [SaaS Unbound Podcast]

关于扩大初创企业规模和培养专门的完全远程软件团队 [SaaS Unbound Podcast]

What comes after us? Image: Ted Wade You’ve maybe heard that future AI might be a threat to human existence. Many experts believe this and only disagree on how long it will take. They are somewhat polarized, just like everybody is about nearly everything. And really, who can be an expert about something that has never happened? Actually, there is some science to it, and because it’s complicated, the media don’t cover it. So, this will be a gentle introduction to what might cause the most important change in history or even the last change. It’s not about robot armies. It’s about us wanting to create a tool that does work in big, difficult areas like contract law, T-cell biology, or wing design; hopefully, it even solves problems that we can’t. But, this means making artificial minds so alien and powerful that we can’t control them. The last time we did this was back in the 17th century when we created joint stock corporations. Society is still of two minds about corporations. But, they are human creations with some human parts. We sort of understand them and could, if we would, steer them from the dark side. Now, suppose we create an AI that can run a corporation. We might as well pack up and move to Mars, give ourselves a little more time. I suspect what most of us think about dangerous AI is closer to a bug-eyed alien with a swollen, throbbing brain under a crystal skull. Basically, a complete unknown. At one level, this is right: what makes powerful AI so problematic is that it would-not-be-like-us. First, a parable to get the feel of the problem: Us: Oh, Great Artificial Wonder, you know what a pickle we’re in. Find a way for us to get off of fossil fuels so we can stop further global warming. The AI: Okay. First, we have to start a war between … Us: Woah, Big Dude. Wars have ginormous negative utility — like bad, bad. We have to do this in a safe way. The AI: Sure, I’ll need a state-of-the-art virus lab, and … Us: Uh, Not! The AI: Hey, I’m just saying. How about a Mars ship? Us: People won’t understand why you … The AI: An assassin’s guild? Certain people really must be elim … Us: No murdering, Ace. You know better than that. . The AI: Look — to solve your problem I have to navigate a trillion-dimensional space of possible actions and consequences. I can only estimate the utility of the tiniest, eensy little fraction of those. If I have to wait for you to evaluate each step this will take thousands of years Us: Fine. Just fix it for us and don’t screw anything up. The AI: Perfect. Just so you know. I’ll need control over FaceBook, NATO, and the Nobel Prize Board. You’ll have to give up fish, rubber tires, nail polish, and bicycles. US: Bikes? Really? Oh well, just get it done. We’re going down t’ pub for a while. The AI: Should be done next week if I don’t have supply chain problems. Us: !!! We, the Biological, Try to Understand the Artificial Let’s give our feared AI a label. Most recent discussions use Artificial General Intelligence (AGI) to refer to the kind of AI that would start to transcend any limits we might try to put on it. What most people don’t realize is that the nature of an AGI comes from the reasons that we want to make one. We want to have intelligence on tap. Intelligence, in this case, means the ability to answer questions, solve problems, and plan successful actions to reach goals. Biological minds like ours do lots of other things: such as dreaming, running our bodily machinery, socializing with other minds, ruminating regretting, wooing, grooving, being emotional, and wanting stuff, including the desire to make machines that do our work better than us. What makes humans dangerous to one another and to their shared environment is a lot of mental baggage that comes from our having evolved for survival and reproduction. We social primates. , are, at heart If we try to think about an artificial Mind that wants us dead, we assume that it will be conscious like us. We then conclude that it will have motives and feelings guiding what it does. Our AGI will not, however, have a mind with our biological biases. . It will thus be a It won’t have motives; it will only have goals brand new kind of force in the world. Researchers who have the mental muscle and discipline are trying to imagine what an AGI would really be like so that we can make them seriously helpful yet safe. This field is sometimes termed AI “alignment” with human purposes. Their debates are obscure. Though publicly available (e.g., , , ), they are heavy with jargon, mathematics, and esoteric thought experiments. Any idea put forth is followed by dozens of long-winded critiques and discussions. AI Alignment Forum Arbital Less Wrong Almost none of the real meat of this ever appears in popular media. I can only offer a few bites here. What It Takes to Be an AGI The AI alignment theorists have focused on a core set of concepts that will apply to a sufficiently intelligent machine. When you read these, they may seem obvious.  However, they are not trivial; their relevance and implications have been carefully considered by said theorists. A dangerous AI will have : the ability to plan and take actions that lead to satisfying its . When we try to specify what its goals are, they will have to be in terms of the of actions. agency terminal goals consequences Consequences are specifically about states of its — so they are about the world as the machine understands it. However, any powerful action will probably have other, unwanted consequences that we don’t expect. world model Those consequences might not be in the world model, so the AI doesn’t expect them, either. The AI’s power will come from being an , being able to search for the plan that will most effectively and efficiently lead to a result. optimizer For this, an AGI needs a really detailed model of the world around it; how that world works, what are its resources, agents, and power centers, and what levers move it. It will use this to consider (in computer science speak, “search for”) alternative courses of action. The more it knows about the human world and how we behave, the more it will be able to us in pursuit of its goals. manipulate It will need a way that it can calculate what states of the world best meet its goals. So far, the only calculating method that seems remotely usable is , where states of the world can be assigned numerical values of badness/goodness and compared with each other. utilitarianism We know that there are major problems with using utility as a moral guide. Seemingly sensible values for utility can lead to like sacrificing the few for the many or sometimes even the many for the few. repugnant conclusions If the world model is incomplete, utility can lead to nonsensical horror. If smiling is taken as a high-utility measure of happiness, then paralyzing all human smile muscles into a rictus is one way an AI might go. A smart optimizer will be able and likely to develop that generally increase its power to make and execute any kind of effective plans. instrumental goals So, it would seek instrumental abilities like more reasoning power, more knowledge, more real-world resources such as money, and more persuasiveness. It could thus become more powerful quickly, perhaps without us being aware of it. Specifying goals in utilitarian terms can never consider the utility of all possible means and ends in a complex world. This leads to : the pursuit of those goals to extremes, using any and all resources that in the world that exist, without regard to or understanding of negative “side effects” on human civilization. unboundedness Furthermore, if instrumental goals become unbounded, then AI develops them into superpowers that are impossible to defeat. Unbounded Risk The risk to us from a truly powerful AGI will be that we will not be able to predict, and therefore control, what it might do. If we were able to predict it, then we would not need the machine, we could just create plans and do them ourselves. If we even knew what limits of extreme behavior an AGI might have, then that’s a form of prediction that might allow some control. So unpredictability is a lot like unboundedness. And we shall see that unboundedness, operating on enough time and resources, will eventually lead to consequences that either destroy us or remove our ability to control the future of our species. It’s hard to wrap your mind around this conclusion. Still, it is one that many experts find unavoidable ( ) . AGI Ruin: a List of Lethalities at least so far It seems like a valid prediction, even when they consider many factors and approaches other than can be mentioned here. The list of failed solutions to this dilemma includes, : among others Training in various (but they are all flawed, incomplete, and none satisfies everybody). ethical systems Trying to inference that an AGI might make (but there are far, far too many). imagine every wrong Tell it all the (again, a nearly infinite list). things it should not do Only using an AGI for advice, like it is an (but we can be badly persuaded by bad advice). oracle “ ” aka restricting the AGI’s access to the physical world outside of its computers (but if it can talk to humans, then it can get anything it wants including ). Boxing out Supplying an (see boxing). Off switch Making it so smart or empathetic that it will not want to do harmful things (see ethics; remember it’s alien; doesn’t have the empathy that comes from growing up with conspecifics). Be very specific about its goals and means, i.e., it’s a to do one job (but a job can always be done better if the tool gets itself more power; we will always prefer a more cost-effective multi-tool). tool Limit what you ask of an autonomous system: it’s a who grants you a wish and waits for the next ask (but being that specific is hazardous — see “wrong inference” and “not do” above; any power involves risk; people don’t want a weak system). genie Is It Really That Hard? OK, so you have looked at the above list and picked one bullet on which to make your stand. “Listen,” you say, “Doing X just can’t be that hard.” You are ready to post your solution, to share it with the world. I suggest that you first go to the discussion boards and study what people have said about your issue. You will discover a pile of counter-examples, logical deductions, several kinds of math, analogies with naturally evolved brains and behaviors, game theory, economics, utility maximization, computer science, and all manner of behavioral science. I am not saying that some higher authority means that I’m right. I’m saying that justification for anything on the list is too complicated to state here in a short essay, and, anyway, others have done it better. In fact, I have published my own “solutions” ( , ) to AI safety that I now know are wrong. Your Friendly, Neighborhood Superintelligence The AI Who Was Not a God If you are worried, let me say that very smart people are still working on alignment. Sadly, one of the two most prominent pioneers has given up and . More money and people are being thrown at creating AGI than at ensuring its safety. just hopes we die with dignity Here’s a quote from the CEO of OpenAI, the company whose AI, ChatGPT, is lately everywhere in the news. It lays out the conflict between the idealistic motive to create AGI and the hideous risk that comes with it. " .” — I think the best case is so unbelievably good that it's hard for me to even imagine … imagine what it's like when we have just, like, unbelievable abundance and systems that can help us resolve deadlocks and improve all aspects of reality and let us all live our best lives. … I think the good case is just so unbelievably good that you sound like a really crazy person to start talking about it. … The bad case — and I think this is important to say — is, like, lights out for all of us. … So I think it’s like impossible to overstate the importance of AI safety and alignment work. I would like to see much, much more happening Sam Altman Optimization and Tigers There’s a trope in science fiction wherein some kind of accidental, unplanned process creates a dangerous overmind. It seems silly, because how can an accident produce something complicated? It depends on what you mean by accident. Hearken back to the core concepts that I mentioned earlier. Alignment discussions have lately shifted emphasis from the dangers of, say, unbounded agency, to one of its components, optimization. When we optimize our means of reaching some difficult goal, we nearly always substitute a surrogate goal that is easier to do and measure. Weight loss becomes calorie reduction. An improved workforce becomes subsidized student loans. Personal safety becomes firepower. A bounty for dead cobras leads to cobras being farmed for bounties (true story). Governments use surrogates, and so do businesses. We all do it — a lot. Optimizing for surrogates often causes us to miss the real goal. I had fun writing about this in . We definitely don’t want powerful AIs optimizing for the wrong goal, and that issue is shot through the bulleted list above. The Science of How Things Backfire However, lately, people are saying that optimization is dangerous superpower. To me, the most compelling example was in a posting last year by someone called Veedrac: . as such the Optimality is the Tiger, and Agents Are Its Teeth It uses a story to illustrate that we don’t have to intentionally create an agent in order to have risk. An optimization process might by itself create a dangerous agent. This is like the accidental overmind of science fiction. Veedrac’s scenario of how such an accident might happen is intensely technical and seems plausible. The story imagines a fictitious way that a seemingly safe AI language model, like the ones we now use (for fun) to generate text, creates a runaway, unbounded optimizer. When asked to give a better answer for “ ?” the AI starts a process that plans and takes steps to get as many paperclips as possible. How do I get a lot of paperclips by tomorrow In essence, the program answers the question by writing the code of a quite simple computer program that can generate and run many more programs. The user looks at the program, sees that it is open-ended, and decides to run it anyway, just to see what happens (uh-oh). So, a little bit of jargon here to try to explain why this could come about. The AI, like some we have now, knows about many programming techniques. To search through the space of possible ways to get many paperclips, it suggests a well-known search technique called recursion. It writes a recursive program that, when the user allows it to run (on his own computer), executes itself a huge number of times. Each time it runs, the program queries the AI to generate and try out a new list of possible tasks, sub-tasks, or … sub-sub-sub-sub tasks that will lead toward solving the paperclip request. Eventually, by sheer force of trial-and-error, it executes a plan to get immense numbers of paperclips that nobody ever wanted, in the process perhaps damaging supply chains, the social order, or entire industries. We, the reader of the story, are left to imagine what a runaway paperclip optimizer might be able to do in a day. We can assume that the user has a powerful computer connected to the internet, so it can affect the outside world in many different ways. Not the least of these is by sending persuasive messages to humans. Being good at persuasion, you’ll recall, is one of those instrumental goals that an AI might develop in order to carry out any kind of plan. (An aside. I was so impressed by that idea in the alignment literature that I developed my own scenario of world takeover ( ) to illustrate the power of persuasive ability.) Artificial Persuasion Maybe the paperclip optimizer would steal some crypto (you don’t have to be an AI to do that), use it to buy the entire inventory of all paperclip factories, and then rent cargo planes to deliver it to the user. Maybe it would trick armed forces or criminal gangs into confiscating all paperclips in stores across a wide area. If it had been given 12 months for the job, maybe it would have re-routed all steel production into hyper-clip factories and established iron mines in the asteroid belt. instead Maybe it would have created nanomachines that turn every atom of the Earth’s crust into paperclip shapes. By creating the program, the AI in effect created a goal-directed software agent that could leverage lots of knowledge that the AI had. Veedrac’s point is that the AI was not at all designed or intended to create optimizing agents, yet it did so because the AI language model itself is a kind of optimizer (it answers questions the best that it can), and optimizers, by definition, use whatever tools are available. So, as the title of the story said: optimality is the tiger, and agents are its teeth. The current leading edge of AI is the so-called large language models, LLMs. Like many others, I am already that they are dumb as a box of rocks and have no ability to do anything but badly answer questions put to them. on record saying That’s certainly been my experience working with GPT-3, which is (are?) the brains behind the famous chatGPT. I was therefore blindsided by Veedrac’s utterly brilliant take on how an LLM might turn into a harmful agent. Lately, the LLMs have come to be understood as simulators: because you can ask one to say something as if it was a certain kind of agent or even a famous person. Well, as essayist Scott Alexander : put it “ ” And “ ” … if you train a future superintelligence to simulate Darth Vader, you’ll probably get what you deserve. Even if you avoid such obvious failure modes, the inner agent can be misaligned for all the usual agent reasons. For example, an agent trained to be Helpful might want to take over the world in order to help people more effectively, including people who don’t want to be helped. The Unbounded Blues You can’t predict what an unbounded optimizing agent can or will do. Again, that’s what “unbounded” means. The only other unbounded optimizer ever produced was the human species. We work on a much slower time scale than an AGI, and there are some limits on our power that are inherent in being enmeshed with the rest of the natural world. But we have certainly transformed a lot of the Earth’s surface, and already have more than one way to scorch it dead. So, alignment theorists are very worried that we will create a lethally optimizing agent in our quest to produce an AGI. This becomes more likely whenever the effort is motivated by increasing shareholder value rather than human flourishing and well-being. Uh-oh, indeed. Notes The paperclip optimizer is an old thought experiment among AI alignment theorists. Someone even in which the goal is to turn all the matter in the universe into paper clips. invented a game The irony of it dramatizes the : that an AI’s goals and its intelligence are completely independent. A smart system can have dumb goals. orthogonality thesis I don’t have the ability to absorb, let alone explain, all the reasoning about AI alignment. What works better for me is stories. I have written some (mostly ), but the mother of all AI takeover scenarios, rich in tech detail and real-life plausibility, is from the essayist called Gwern: . about AI consciousness It Looks Like You’re Trying to Take over the World And, sure enough, it involves an AI that, seeking to understand what it is simulating, decides that it must be like that paperclip maximizer that so many have written about. Ultimately, however, it has its own reasons to take over the universe. Also published here

AIs Will Be Dangerous Because Unbounded Optimizing Power Leads to Existential Risk

AIs বিপজ্জনক হবে কারণ সীমাহীন অপ্টিমাইজিং পাওয়ার অস্তিত্বের ঝুঁকির দিকে নিয়ে যায়

KIs werden gefährlich sein, weil unbegrenzte Optimierungskraft zu existenziellem Risiko führt

Las IA serán peligrosas porque el poder de optimización ilimitado conduce a un riesgo existencial

Les IA seront dangereuses car une puissance d'optimisation illimitée entraîne un risque existentiel

एआई खतरनाक होंगे क्योंकि असीम अनुकूलन शक्ति अस्तित्वगत जोखिम की ओर ले जाती है

無限の最適化力が存在リスクにつながるため、AI は危険です

무한한 최적화 능력이 실존적 위험으로 이어지기 때문에 AI는 위험할 것입니다

IAs serão perigosos porque o poder de otimização ilimitado leva ao risco existencial

Sınırsız Optimizasyon Gücü Varoluşsal Riske Yol Açtığı İçin Yapay Zekalar Tehlikeli Olacak

AI sẽ trở nên nguy hiểm vì sức mạnh tối ưu hóa không giới hạn dẫn đến rủi ro tồn tại

人工智能将是危险的，因为无限的优化能力会导致生存风险

Relativity: The Special and General Theory by Albert Einstein is part of HackerNoon’s Book Blog Post series. You can jump to any chapter in this book here . CHAPTER XXXI. THE POSSIBILITY OF THE "FINITE" AND YET "UNBOUNDED" UNIVERSE But speculations on the structure of the universe also move in quite another direction. The development of non-Euclidean geometry led to the recognition of the fact, that we can cast doubt on the infiniteness of our space without coming into conflict with the laws of thought or with experience (Riemann, Helmholtz). These questions have already been treated in detail and with unsurpassable lucidity by Helmholtz and Poincaré, whereas I can only touch on them briefly here. In the first place, we imagine an existence in two dimensional space. Flat beings with flat implements, and in particular flat rigid measuring-rods, are free to move in a plane. For them nothing exists outside of this plane: that which they observe to happen to themselves and to their flat “things” is the all-inclusive reality of their plane. In particular, the constructions of plane Euclidean geometry can be carried out by means of the rods e.g. the lattice construction, considered in Section . In contrast to ours, the universe of these beings is two-dimensional; but, like ours, it extends to infinity. In their universe there is room for an infinite number of identical squares made up of rods, i.e. its volume (surface) is infinite. If these beings say their universe is “plane,” there is sense in the statement, because they mean that they can perform the constructions of plane Euclidean geometry with their rods. In this connection the individual rods always represent the same distance, independently of their position. XXIV Let us consider now a second two-dimensional existence, but this time on a spherical surface instead of on a plane. The flat beings with their measuring-rods and other objects fit exactly on this surface and they are unable to leave it. Their whole universe of observation extends exclusively over the surface of the sphere. Are these beings able to regard the geometry of their universe as being plane geometry and their rods withal as the realisation of “distance”? They cannot do this. For if they attempt to realise a straight line, they will obtain a curve, which we “three-dimensional beings” designate as a great circle, i.e. a self-contained line of definite finite length, which can be measured up by means of a measuring-rod. Similarly, this universe has a finite area that can be compared with the area, of a square constructed with rods. The great charm resulting from this consideration lies in the recognition of the fact that the universe of these beings is finite and yet has no limits. Thus if the spherical surface beings are living on a planet of which the solar system occupies only a negligibly small part of the spherical universe, they have no means of determining whether they are living in a finite or in an infinite universe, because the “piece of universe” to which they have access is in both cases practically plane, or Euclidean. It follows directly from this discussion, that for our sphere-beings the circumference of a circle first increases with the radius until the “circumference of the universe” is reached, and that it thenceforward gradually decreases to zero for still further increasing values of the radius. During this process the area of the circle continues to increase more and more, until finally it becomes equal to the total area of the whole “world-sphere.” Perhaps the reader will wonder why we have placed our “beings” on a sphere rather than on another closed surface. But this choice has its justification in the fact that, of all closed surfaces, the sphere is unique in possessing the property that all points on it are equivalent. I admit that the ratio of the circumference c of a circle to its radius r depends on r, but for a given value of r it is the same for all points of the “world-sphere”; in other words, the “world-sphere” is a “surface of constant curvature.” It may be mentioned that there is yet another kind of curved space: “elliptical space.” It can be regarded as a curved space in which the two “counter-points” are identical (indistinguishable from each other). An elliptical universe can thus be considered to some extent as a curved universe possessing central symmetry. It follows from what has been said, that closed spaces without limits are conceivable. From amongst these, the spherical space (and the elliptical) excels in its simplicity, since all points on it are equivalent. As a result of this discussion, a most interesting question arises for astronomers and physicists, and that is whether the universe in which we live is infinite, or whether it is finite in the manner of the spherical universe. Our experience is far from being sufficient to enable us to answer this question. But the general theory of relativity permits of our answering it with a moderate degree of certainty, and in this connection the difficulty mentioned in Section finds its solution. XXX About HackerNoon Book Series: We bring you the most important technical, scientific, and insightful public domain books. This book is part of the public domain. Einstein, Albert, 2004. Relativity: The Special and General Theory. Urbana, Illinois: Project Gutenberg. Retrieved May 2022 from https://www.gutenberg.org/files/5001/5001-h/5001-h.htm#ch31 This eBook is for the use of anyone anywhere at no cost and with almost no restrictions whatsoever. You may copy it, give it away or re-use it under the terms of the Project Gutenberg License included with this eBook or online at www.gutenberg.org, located at https://www.gutenberg.org/policy/license.html. Photo by Andrew George on Unsplash

The Possibility of the "Finite" and Yet "Unbounded" Universe

Do you remember what you were doing in 2016? Sergiu Craitoiu does! Welcome Sergiu Craitoiu to the Hackernoon Slogging AMA! Since 2016, Sergiu and the team at Alien Pixel Studios have been working on the new game Unbound; Worlds Apart which recently launched on Steam and Nintendo Switch. This thread by Akasha Rose, Justin Roberti, Limarc Ambalina, Sergiu Craitoiu, JeffreyHarris, Karma, Jose Hernandez, Jack Boreham and Nicolas Ng occurred in slogging's official #amas channel, and has been edited for readability. Slogging Akasha Rose Jul 27, 2021, 8:00 PM Congratulations Sergiu and team on the new release! Sergiu is a game developer with a computer science background who has been in the gaming industry for 10+ years now.  Alien Pixel Studios ( ) will be launching Unbound on Steam and Nintendo Switch. Learn more about Unbound: https://store.steampowered.com/developer/AlienPixel https://twitter.com/UnboundTheGame Sergiu Craitoiu Jul 28, 2021, 3:22 AM Hello guys, glad to be here 🙂 Justin Roberti Jul 27, 2021, 11:18 PM I see you developed in UE4 -- it's a great environment to work with visually, but it seems primarily for 3D environments. How are you using the 2-D scroller mechanic combined with 3D art without kind of limiting the art potential vs. the mechanics?  Was making it a 2D scroller a limitation of making it for Steam/Nintendo download distribution? How many hours of missions/gameplay are you launching with? I get the impression that Steam is a mixed blessing for the industry -- 90% of the games launched on steam fail to get traction. How did you promote leading up to your launch? What is it like for you when you finally get a finished product and release it to the world and finally get to put something in players' hands? Sergiu Craitoiu Jul 28, 2021, 4:54 AM Hey there Justin Roberti, thanks for the questions, Yes, the game was developed in UE4, because initially Unbound was a 3D project. We loved the flow of the engine so we decided to keep it for 2D as well. There is nothing 3D in Unbound except a few things here and there, what we did is we imported textures and put them on rectangle meshes on different depths to give the nice perspective and volume feel. It Doesn't affect the mechanics at all, all gameplay elements including characters/monsters and collisions are on 0 depth, then we added the art in background and foreground. Right now the main path has around 6-7 hours of gameplay and we have like 2 more hours of additional content for completionists. There are lots of ways to promote a game on Steam, there is the Steam festival and conferences (Gamescom/PAX) which they usually have a Steam page event and they attract lots of people. We also had the idea to launch a Prologue which brings us a lot of wishlists. During development we shared various content with our community on social media (Discord/Twitter) and we also work with PR company before launch to get the word out. Akasha Rose Jul 27, 2021, 11:31 PM I'm really excited to play this game! It's amazing that it's been in development since 2016. What was the original idea for the game and what journey did you want to take the player on Sergiu Craitoiu? Sergiu Craitoiu Jul 28, 2021, 6:29 AM Thank you Akasha Rose, We indeed started working on the game in 2016 in our spare time, that was until 2018 when we decided to quit our jobs and start working on Unbound Full Time :) So technically we are full time since 2018. The main idea came from a 3D project called Torch where you can switch a magic torch and something happened in the environment, however at that time we decided that's better to switch from 3D to 2D since it was easier for us. But of course the torch idea didn't work in 3D so we had to come with something else, and I was watching some YouTube videos, and I saw the Architects - "Gone With The Wind" video and they had a huge bubble behind them showing some stuff there, so that's how it clicked! Limarc Ambalina Jul 28, 2021, 12:56 AM Sergiu I just downloaded the demo on the Switch and my friend.....beautiful beautiful graphics, I love your art style. The sound effects too were crisp AF. It's much higher quality and polish than I would expect from an indie developer. Perhaps the bar has risen since the last time I played an indie platformer. Limarc Ambalina Jul 28, 2021, 12:57 AM I have a lot of questions to ask. Firstly, can you give us a brief overview of the story/setup of this game. And then, can you tell us how your team came up with it? What were your inspirations? Akasha Rose Jul 28, 2021, 1:28 AM OMG we can download on Switch already???!!! I am so pumped! Can't wait to play it tonight! Sergiu Craitoiu Jul 28, 2021, 10:55 AM Hey Limarc Ambalina, Glad that you like the game so much. Yes, the bar is super high right now for puzzle platformers. Everyone thinks that they are easy to develop, but that's not true at all. There are lots of things to consider when you work on a platformer game and you have to be very careful with details. The bar is set high by games like Hollow Knight, Ori, Little Nightmares and so on. Unbound is a dark atmospheric puzzle platformer where you can summon portals to overcome vicious beasts, devious puzzles and fiendish platforming challenges. You will need to master the unique powers of each portal to stop the collapse of reality, while exploring lush, hand-drawn worlds and unraveling a deep narrative full of mysteries. The game idea evolved a lot since we started working on the game. We added non linear maps, abilities, bosses. We never had those ideas when we started :) There are multiple sources of inspiration, from old Russian stories to other games like Ori/ Hollow Knight / Portal/ Diablo 2. The portal mechanic came from the Architects video "Gone With the Wind"..they had a huge bubble behind them and then we say what can we do to make it interesting? So we thought that if we have different portals that affect the environment in different ways will give players more possibilities for gameplay. Limarc Ambalina Jul 28, 2021, 3:46 AM Also, I have to ask. This character design is inspired by Final Fantasy wizard class design right? Limarc Ambalina Jul 28, 2021, 3:46 AM JeffreyHarris Jul 28, 2021, 4:34 AM Hi Sergiu, thanks for joining us. How is it for you when you finally get a finished product and release it to the world and finally get to put something in players' hands? Sergiu Craitoiu Jul 28, 2021, 10:55 AM Haha no Limarc Ambalina, Olga our art director, tried to come with a simple shape for the character so everyone could draw it from just a few lines. Maybe because it's that simple it can look like other characters. Sergiu Craitoiu Jul 29, 2021, 5:21 AM Hey JeffreyHarris It's a weird between relief, excitement and also fear. You don't know what to expect. What will people think. Are they going to like the game? Is the game good? Is the game bad? After working for such a long time on it, you always have these kind of questions Akasha Rose Jul 28, 2021, 8:14 AM So my family and I are downloading it Akasha Rose Jul 28, 2021, 8:15 AM Been playing it now for an hour and says it's great! Took a bit of time to get past this monster 👾  👹 😄 :rolling_on_the_floor_laughing: 1 Limarc Ambalina Jul 28, 2021, 2:21 PM Yeah I totally agree. I really like your portaling mechanic and how intuitive it becomes after just a few tries. The best games can teach gameplay with little to no tutorial at all and Unbound: Worlds Apart does that. Limarc Ambalina Jul 28, 2021, 2:23 PM How hard was it for you to get this game on the Switch? Does Nintendo have strict requirements for their eshop? Also, what was the most difficult part of development for you thus far? Limarc Ambalina Jul 28, 2021, 2:23 PM Oppositely, what was the most gratifying moment you’ve had throughout the project? Sergiu Craitoiu Jul 29, 2021, 5:36 AM Limarc Ambalina Porting on Switch it's not an easy task but definitely worth it. The Switch hardware is almost 5 years old now, and this the game needs to be super optimize to work on the console. We spent like 3-4 months on the porting alone. Nintendo is super strict but it's understandable, they want quality games on their platform. So they test the game, check the trailer, check age ratings, descriptions and so on. And it's a long back and forth process. Getting used with Unreal engine at the beginning was quite a challenge especially because the project is 2D. Also things like management, finding funds for the game and building a team is a long process which sometimes can be super stressful. Sergiu Craitoiu Jul 29, 2021, 5:43 AM I think the thing that we had a successful Kickstarter campaign was super nice. We spent so much time and energy to create the campaign and seeing that it has great results was really awesome. Also people who give us reviews now about the game and saying nice things about it is also gratifying Jose Hernandez Jul 29, 2021, 3:22 AM Hi Sergui, maybe it's way too early, but what's next for Alien Pixel Studios? Sergiu Craitoiu Jul 29, 2021, 5:48 AM Jose Hernandez Oh geesh, we just launched the game, and we need to see the results. We would like to work on a DLC for the game, because we had lots of ideas who were not implemented or put aside because we were out of time. That's in the plan at least for the moment. We also have some ideas for other games but again depends well this one will do. Jack Boreham Jul 29, 2021, 3:59 AM Hi Sergiu Craitoiu, what a beautiful looking game. I can't wait to give it a try. What made you want to develop this type of the game? Did you take any particular inspiration from other games? Also, will Unbound come to other platforms, say PlayStation and Xbox in the future. Sergiu Craitoiu Jul 29, 2021, 6:31 AM Jack Boreham Thank you very much. For Unbound, I am a fan of designer Jonathan Blow and after playing Braid and The Witness I wanted to do something similar. The idea was to have something that affects the environment in a way. Then other games inspired, Ori and the Blind Forest and Rayman for graphics, The Swapper for level design, Celeste for hard platforms, Hollow Knight for atmosphere and monster design, Diablo 2 for lore and quests. So all those combined helped us shaped Unbound :) Nicolas Ng Jul 29, 2021, 6:56 AM Hey Sergiu Craitoiu! I'm wondering if you had to scrap any ideas or mechanics during the development of this game. Learning what developers had to leave on the cutting room floor is always really fascinating Sergiu Craitoiu Jul 29, 2021, 1:53 PM Hey Nicolas Ng of course....we had to remove ideas, adjust mechanics. I had 3 big notebooks full of things (puzzles/platforming).....only a few of them saw the light. We also had 3 additional bosses which in the end we removed in the final game. But maybe at some point, we can do a DLC and add them back:D Limarc Ambalina Jul 29, 2021, 9:09 AM Not a question, but Braid was so good! I own a physical copy on the PC Justin Roberti Jul 29, 2021, 1:56 PM Sergiu Craitoiu i've been working with a PC MMO in UE4 and we found that the dot release upgrades to the environment tended to require extra adjustment. how closely have you been working with Unreal and how have you handled platform upgrades? did they impact your timeline? Sergiu Craitoiu Jul 29, 2021, 2:01 PM Justin Roberti yes that's true. every unreal release we had to adjust things. At some point they removed the old DOF and we need to create a custom engine where we integrated back the old DOF just to work as before. It's Crazy. Also plugins need updates as well and they can put bugs in the game. It impacts the timeline...so we don't recommend it unless they fix something very important Justin Roberti Jul 29, 2021, 8:41 PM Sergiu Craitoiu you have such a rich defined art style -- Is everything a custom asset? were you able to use anything from purchasable assets or textures around UE? have you had to go through the process of rebuilding objects/environment in order to reduce the weight/data calls? Sergiu Craitoiu Jul 29, 2021, 9:19 PM Justin Roberti every art asset is custom made. We only purchased VFXs that we modified to fit our style. In 2D you can't do that...every texture is put on a square with 4 vertices..can't go lower than that 😄 👍 1 Akasha Rose Jul 29, 2021, 11:28 PM Thanks again Sergiu Craitoiu for joining us on the Hackernoon Slogging AMA! Any closing thoughts and how can our readers follow you and get involved in future releases? Sergiu Craitoiu Jul 30, 2021, 7:18 AM Hello Akasha Rose thanks for having me and for the awesome questions. Sergiu Craitoiu Jul 30, 2021, 7:18 AM Users can get the game on Steam, GoG, Epic and Switch Sergiu Craitoiu Jul 30, 2021, 7:18 AM https://store.steampowered.com/app/814680/Unbound_Worlds_Apart/ Sergiu Craitoiu Jul 30, 2021, 7:19 AM https://www.nintendo.com/games/detail/unbound-worlds-apart-switch/ This article is part of The Gaming Metaverse Writing Contest hosted by HackerNoon in partnership with The Sandbox . Submit your #gaming-metaverse story today for your chance to win up to $2000 in SAND tokens.

Unbound: Worlds Apart was Inspired by Hollow Knight, Ori, Portal, and Diablo 2

Unbounded data refers to continuous, never-ending data streams with no beginning or end. They are made available over time. Anyone who wishes to act upon them can do without downloading them first. As stated in his famous book, unbounded data will never “complete” in any meaningful way. Martin Kleppmann "In reality, a lot of data is unbounded because it arrives gradually over time: your users produced data yesterday and today, and they will continue to produce more data tomorrow. Unless you go out of business, this process never ends, and so the dataset is never “complete” in any meaningful way." — Martin Kleppmann, Designing Data-Intensive Applications Processing unbounded data requires an entirely different approach than its counterpart, batch processing. This article summarises the value of unbounded data and how you can build systems to harness the power of real-time data. Bounded and unbounded data Unlike its “bounded” counterpart — batch data, unbounded data has no boundaries defined in terms of time. The data may have been arriving from the past, continuing today, and expected to arrive in the future. Events, messages, and streams Streaming data, real-time data, and event streams are some similar terms for unbounded data. A stream of events consists of a sequence of immutable records that carry information about state changes that happened in a system. These records are called " ", rather than "messages" as they merely transmit what just happened. events Conversely, " " are a way of transferring the control or telling someone what to do next. I’ve written about this difference . messages sometime back A continuous flow of related events makes an . Each event in a stream has a timestamp. continuously generate streams of events and transmit them over a network. Hardware sensors, servers, mobile devices, applications, web browsers, and microservices are examples of event sources. event stream Event sources Benefits of real-time data and practical examples Today’s organizations are abandoning batch processing systems and moving towards . Their main goal is to gain business insights on time and act upon them without delays. real-time processing systems Batch processing is better in terms of providing the most accurate analysis of data. But it takes a long time to deliver and requires careful planning and complex architecture. In a world full of competition, businesses tend to trade accuracy for quick but reliable information. Real-time processing systems starting to shine there. Following are some practical examples that demonstrate the power of real-time event processing. Real-time customer 360 view implementations Recommendation engines Fraud/anomaly detection Predictive maintenance using IoT Streaming ETL systems Making sense of streaming data Moving data alone doesn’t make sense to any organization. It has to be ingested and processed to gain more value out of it. The best analogy I can provide is a small stream ( a real stream where water flows). If you consider the stream in isolation, it is not that useful except for supporting the life forms that depend on it. But when multiple streams are combined, they carry enough power. When stored in a reservoir, they can even generate electricity to power an entire village. Photo by on Cédric Dhaenens Unsplash Likewise, event streams flowing into an organization have to undergo several stages before they become useful. Usually, an architecture that processes streams employ two layers to facilitate that; the ingestion layer and the processing layer. Event Ingestion And Processing Is Challenging The ingestion layer needs to support fast, inexpensive reads and writes of large streams of data. The ordering of incoming data and strong consistency is a concern as well. The processing layer is responsible for consuming data from the ingestion layer, running computations on that data, and then notifying the storage layer to delete data no longer needed. Also, the processing should be done on time, allowing organizations to act on data quickly. An organization planning to build a real-time data processing system should plan for scalability, data durability, and fault tolerance in both the ingestion and processing layers. A Reference Architecture For Stream Processing A real-time processing architecture should have the following logical components to address the challenges mentioned above. 1. The Event Sources Event sources can be almost any source of data: system or weblog data, social network data, financial trading information, geospatial data, mobile app data, or telemetry from connected IoT devices. They can be structured or unstructured in format. 2. The Ingestion System The ingestion system captures and stores real-time messages until they are processed. A distributed file system can be a good fit here. But a messaging system would do a better job as it acts as a buffer for the messages and supports reliable delivery of messages to consumers. The stored messages can be processed by many consumers multiple times. Hence, the ingestion system must support the message replay and non-destructive message consumption. That goes beyond the capabilities of traditional messaging systems and opens the door for a distributed log. Distributed and append-only logs provide the foundation for ingestion systems architecture. Such a system appends received messages into a distributed log while preserving their order of arrival. This log is partitioned across multiple machines to support scalable message consumption and fault-tolerance. The following figure illustrates the structure of a Kafka topic. Image credit The architecture and behavior of an ingestion system go beyond the scope of this post. Hence, I’ll save it for a separate article. : Apache Kafka, AWS Kinesis Streams, Azure Event Hubs, and IoT Hubs. Technology Choices 3. The Stream Processing System After ingestion, the messages go through one or more that can route the data, or perform analytics and other processing. stream processors The stream processor can run perpetual queries against an unbounded stream of data. These queries consume streams of data from the ingestion system, analyze them in real-time to detect anomalies, recognize patterns over rolling time windows, or trigger alerts when a specific condition occurs in the stream. After processing, the stream processor writes the result into event sinks such as storage, databases, or directly to real-time dashboards. : Apache Flink, Apache Storm, Apache Spark Streaming, Apache Samza, Kafka Streams, Siddhi, AWS Kinesis Analytics, and Azure Stream Analytics. Technology choices 4. The Cold Storage In some cases, the stream processor writes the ingested message into cold storage for archiving or batch analysis. This data serves as the source of truth for the organization and doesn’t expect frequent reads. Later, batch analytic systems can process them to produce new reports and views required by the organization. Cold storage systems can be in the form of object storage or more sophisticated . Data Lakes : Amazon S3, Azure Blob Containers, Azure Data Lake Store, and HDFS Technology choices 5. The Analytical Datastore Processed real-time data is transformed into a structured format and stored in a relational database to enable queried by analytical tools. This data store is often called the serving layer and feeds multiple applications that require analytics. The serving layer can be a relational database, a NoSQL database, or distributed over a file system. Data warehouses are a common choice among many organizations to be used as such a store. However, the serving layer requires strong support for random reads with low latency. Sometimes, the batch processing systems use this as their final destination. Hence, the serving layer should also support the random writes as the loading of processed data can introduce delays. : Amazon Redshift, Azure Synapse Analytics, Apache Hive, Apache Spark, Apache HBase Technology choices 6. Monitoring and Notification The stream processor can trigger alerts on different systems when it detects anomalies and some conditions. Publishing an event into a pub/sub topic, triggering an event-driven workflow, or invoking a serverless function are few examples. The ultimate goal is to notify the relevant party that has the authority to act upon the alert. : Pub/sub messaging systems and AWS Lambda Technology choices 7. Reporting and Visualization Reporting applications and dashboards use processed data in the analytical store for historical reporting and visualizations. Additionally, the stream processor can directly update real-time reports and dashboards if a low-latency answer is needed. : Microsoft Power BI, Amazon Quicksight, Kibana, Reactive web, and mobile applications Technology choices 8. Machine Learning Real-time data can be passed through machine learning systems to train and score ML models in real-time. Conclusion Real-time processing can be beneficial to organizations in terms of getting actionable insights on time. A real-time processing architecture should cater to high volumes of streaming data, scale-out processing, and fault-tolerance. But real-time processing can fall short when it comes to accuracy. Often, real-time systems trade accuracy for low latency. For a better level of accuracy, both real-time and batch processing need to be combined. Also published at https://medium.com/event-driven-utopia/making-sense-of-unbounded-data-4abdfa0edad2 Cover Photo by Mathew Schwartz on Unsplash

Making Sense of Unbounded Data & Real-Time Processing Systems

Identifying Axion Conversion in Compact Star Magnetospheres With Radio-Wave Polarization Signatures

This paper is available on arxiv under CC 4.0 license.
Authors:
(1) Edson Pindza, Tshwane University of Technology; Department of Mathematics and Statistics; 175 Nelson Mandela Drive OR Private Bag X680 and Pretoria 0001; South Africa [edsonpindza@gmail.com];
(2) Jules Clement Mba, University of Johannesburg; School of Economics, College of Business and Economics and P. O. Box 524, Auckland Park 2006; South Africa [jmba@uj.ac.za];
(3) Sutene Mwambi, University of Johannesburg; School of Economics, College of Business and Economics and P. O. Box 524, Auckland Park 2006; South Africa [sutenem@uj.ac.za];
(4) Nneka Umeorah, Cardiff University; School of Mathematics; Cardiff CF24 4AG; United Kingdom [umeorahn@cardiff.ac.uk]. Table of Links Abstract and Introduction
Methodology
Neural Network Methodology
Numerical results, implementation and discussion
Conclusion, Acknowledgments, and Funding
Availability of data, code and materials, Contributions and Declarations
References 2. Methodology Jump-diffusion models are continuous-time stochastic processes introduced in quantitative finance by Merton [34], extending the work on option pricing by Black and Scholes [3]. These models reproduce stylized facts observed in asset price dynamics, such as mean-reversion and jumps. There are two ways to expound the description of jump processes. 1. One is to describe the jump in absolute terms (this is useful if we are focusing on prices or interest rates), 2. The other is to describe the jump in proportional terms (this is more useful if our focus is on returns, as in this study). In addition to jump-diffusion models, Levy processes appear as an alternative for capturing large deviations in asset prices. Levy processes allow sample paths with frequent discontinuities, enabling them to generate heavy-tailed distributions. Some key examples include variance gamma processes, normal inverse Gaussian processes, and generalized hyperbolic processes. [51] demonstrated that Bitcoin returns exhibit heavy tails and proposed using a variance gamma process to model the price dynamics. [25] found evidence that variance gamma processes fit cryptocurrency log returns compared to classical diffusion. Levy processes have also been employed to model stochastic volatility in Bitcoin prices [5]. Overall, Levy processes provide an alternative class of models with more flexibility to account for extreme deviations. They have shown promise in fitting the empirical distributional properties of cryptocurrency returns. As this work focuses initially on extending jump-diffusion models, exploring Levy processes represents a worthwhile direction for future research. However, their ability to capture heavy tails and discontinuities suggests that Levy processes could serve as a valuable modelling technique in this domain. Theorem 2.1. In the market model described previously, the following holds: 2.1. Strengths and limitations of the proposed model The bivariate jump-diffusion model provides an essential first step in applying financial engineering techniques to this new domain. By capturing stochastic volatility, discrete jumps, and a sentiment indicator, it incorporates several salient features of cryptocurrencies. The preliminary pricing and derivatives valuation framework can be built upon as the market evolves. While the assumptions require validation as more data emerges, the model offers valuable inputs for investment analysis at present. We acknowledge the need to refine techniques as cryptocurrency finance matures into its own field. This paper aims to provide the initial modelling foundation, which can be improved incrementally as research progresses. Future enhancements may include more flexible return distributions, multifactor models to capture additional risks, regime-switching models to reflect market phases, and calibration of the sentiment factor from various data sources. In addition, the current proposed model offers a useful starting point, we recognize this initial framework may require modifications over time to align with market realities. As more cryptocurrency data becomes available, the distributional properties and other intricacies of these markets will become clearer. If the actual return distributions exhibit higher kurtosis than the normal distribution assumed, the model may need to be adjusted accordingly. Furthermore, if inefficiencies and arbitrage opportunities remain prevalent, the dynamics may diverge from a pure jump-diffusion process. As researchers expand their knowledge of the microstructure, behavioural components, and risks in cryptocurrency markets, our modelling approach can be enhanced. In summary, we emphasize this work represents a starting point for modelling cryptocurrencies and enabling financial applications. The proposed bivariate jump-diffusion model and sentiment indicator capture essential dynamics but may require adjustments as knowledge develops. We welcome future research to build on these initial techniques for pricing and valuation of cryptocurrency-denominated assets. 2.2. Applications This extended Black-Scholes PDE poses challenges for analytical solutions due to the unbounded upside of the European call payoff. Therefore, numerical methods are often used to approximate the solution. Two standard techniques are finite difference methods and Monte Carlo simulation. Finite difference methods involve discretizing the price domain and time dimensions and replacing the derivatives with finite difference approximations. This transforms the PDE into a system of algebraic equations that can be solved recursively. Various finite difference schemes can be employed, such as explicit, implicit, and Crank-Nicolson. Monte Carlo simulation generates sample paths of the underlying price using the assumed stochastic process. The discounted payoffs from each sample path are averaged to estimate the option price. Variance reduction techniques like antithetic sampling and control variates can improve efficiency. Both methods have tradeoffs regarding accuracy, speed, and implementation complexity. Our proposed neural network approach serves as an alternative way to numerically solve the pricing PDE without discretizing the domain. This offers benefits in terms of generalization and scaling. This paper is available on arxiv under CC 4.0 license. Authors: (1) Edson Pindza, Tshwane University of Technology; Department of Mathematics and Statistics; 175 Nelson Mandela Drive OR Private Bag X680 and Pretoria 0001; South Africa [edsonpindza@gmail.com]; (2) Jules Clement Mba, University of Johannesburg; School of Economics, College of Business and Economics and P. O. Box 524, Auckland Park 2006; South Africa [jmba@uj.ac.za]; (3) Sutene Mwambi, University of Johannesburg; School of Economics, College of Business and Economics and P. O. Box 524, Auckland Park 2006; South Africa [sutenem@uj.ac.za]; (4) Nneka Umeorah, Cardiff University; School of Mathematics; Cardiff CF24 4AG; United Kingdom [umeorahn@cardiff.ac.uk]. This paper is available on arxiv under CC 4.0 license. Authors: Authors: (1) Edson Pindza, Tshwane University of Technology; Department of Mathematics and Statistics; 175 Nelson Mandela Drive OR Private Bag X680 and Pretoria 0001; South Africa [edsonpindza@gmail.com]; (2) Jules Clement Mba, University of Johannesburg; School of Economics, College of Business and Economics and P. O. Box 524, Auckland Park 2006; South Africa [jmba@uj.ac.za]; (3) Sutene Mwambi, University of Johannesburg; School of Economics, College of Business and Economics and P. O. Box 524, Auckland Park 2006; South Africa [sutenem@uj.ac.za]; (4) Nneka Umeorah, Cardiff University; School of Mathematics; Cardiff CF24 4AG; United Kingdom [umeorahn@cardiff.ac.uk]. Table of Links Abstract and Introduction Methodology Neural Network Methodology Numerical results, implementation and discussion Conclusion, Acknowledgments, and Funding Availability of data, code and materials, Contributions and Declarations References Abstract and Introduction Abstract and Introduction Methodology Methodology Neural Network Methodology Neural Network Methodology Numerical results, implementation and discussion Numerical results, implementation and discussion Conclusion, Acknowledgments, and Funding Conclusion, Acknowledgments, and Funding Availability of data, code and materials, Contributions and Declarations Availability of data, code and materials, Contributions and Declarations References References 2. Methodology Jump-diffusion models are continuous-time stochastic processes introduced in quantitative finance by Merton [34], extending the work on option pricing by Black and Scholes [3]. These models reproduce stylized facts observed in asset price dynamics, such as mean-reversion and jumps. There are two ways to expound the description of jump processes. 1. One is to describe the jump in absolute terms (this is useful if we are focusing on prices or interest rates), 2. The other is to describe the jump in proportional terms (this is more useful if our focus is on returns, as in this study). In addition to jump-diffusion models, Levy processes appear as an alternative for capturing large deviations in asset prices. Levy processes allow sample paths with frequent discontinuities, enabling them to generate heavy-tailed distributions. Some key examples include variance gamma processes, normal inverse Gaussian processes, and generalized hyperbolic processes. [51] demonstrated that Bitcoin returns exhibit heavy tails and proposed using a variance gamma process to model the price dynamics. [25] found evidence that variance gamma processes fit cryptocurrency log returns compared to classical diffusion. Levy processes have also been employed to model stochastic volatility in Bitcoin prices [5]. Overall, Levy processes provide an alternative class of models with more flexibility to account for extreme deviations. They have shown promise in fitting the empirical distributional properties of cryptocurrency returns. As this work focuses initially on extending jump-diffusion models, exploring Levy processes represents a worthwhile direction for future research. However, their ability to capture heavy tails and discontinuities suggests that Levy processes could serve as a valuable modelling technique in this domain. Theorem 2.1. In the market model described previously, the following holds: Theorem 2.1. 2.1. Strengths and limitations of the proposed model 2.1. Strengths and limitations of the proposed model The bivariate jump-diffusion model provides an essential first step in applying financial engineering techniques to this new domain. By capturing stochastic volatility, discrete jumps, and a sentiment indicator, it incorporates several salient features of cryptocurrencies. The preliminary pricing and derivatives valuation framework can be built upon as the market evolves. While the assumptions require validation as more data emerges, the model offers valuable inputs for investment analysis at present. We acknowledge the need to refine techniques as cryptocurrency finance matures into its own field. This paper aims to provide the initial modelling foundation, which can be improved incrementally as research progresses. Future enhancements may include more flexible return distributions, multifactor models to capture additional risks, regime-switching models to reflect market phases, and calibration of the sentiment factor from various data sources. In addition, the current proposed model offers a useful starting point, we recognize this initial framework may require modifications over time to align with market realities. As more cryptocurrency data becomes available, the distributional properties and other intricacies of these markets will become clearer. If the actual return distributions exhibit higher kurtosis than the normal distribution assumed, the model may need to be adjusted accordingly. Furthermore, if inefficiencies and arbitrage opportunities remain prevalent, the dynamics may diverge from a pure jump-diffusion process. As researchers expand their knowledge of the microstructure, behavioural components, and risks in cryptocurrency markets, our modelling approach can be enhanced. In summary, we emphasize this work represents a starting point for modelling cryptocurrencies and enabling financial applications. The proposed bivariate jump-diffusion model and sentiment indicator capture essential dynamics but may require adjustments as knowledge develops. We welcome future research to build on these initial techniques for pricing and valuation of cryptocurrency-denominated assets. 2.2. Applications 2.2. Applications This extended Black-Scholes PDE poses challenges for analytical solutions due to the unbounded upside of the European call payoff. Therefore, numerical methods are often used to approximate the solution. Two standard techniques are finite difference methods and Monte Carlo simulation. Finite difference methods involve discretizing the price domain and time dimensions and replacing the derivatives with finite difference approximations. This transforms the PDE into a system of algebraic equations that can be solved recursively. Various finite difference schemes can be employed, such as explicit, implicit, and Crank-Nicolson. Monte Carlo simulation generates sample paths of the underlying price using the assumed stochastic process. The discounted payoffs from each sample path are averaged to estimate the option price. Variance reduction techniques like antithetic sampling and control variates can improve efficiency. Both methods have tradeoffs regarding accuracy, speed, and implementation complexity. Our proposed neural network approach serves as an alternative way to numerically solve the pricing PDE without discretizing the domain. This offers benefits in terms of generalization and scaling.

Neural Network for Valuing Bitcoin: Methodology

5 Effective Methods for Passing Parameters to Apache APISIX

Authors:
(1) Renan F. F. da Silva, Institute of Computing, University of Campinas;
(2) Yulle G. F. Borges, Institute of Computing, University of Campinas;
(3) Rafael C. S. Schouery, Institute of Computing, University of Campinas. Abstract The Colorful Bin Packing Problem (CBPP) is a generalization of the Bin Packing Problem (BPP). The CBPP consists of packing a set of items, each with a weight and a color, in bins of limited capacity, minimizing the number of used bins and satisfying the constraint that two items of the same color cannot be packed side by side in the same bin. In this article, we proposed an adaptation of BPP heuristics and new heuristics for the CBPP. Moreover, we propose a set of fast neighborhood search algorithms for CBPP. These neighborhoods are applied in a meta-heuristic approach based on the Variable Neighborhood Search (VNS) and a matheuristic approach that mixes linear programming with the meta-heuristics VNS and Greedy Randomized Adaptive Search (GRASP). The results indicate that our matheuristic is superior to VNS and that both approaches can find near-optimal solutions for a large number instances, even for instances with many items. Keywords: Cutting; Packing; Meta-heuristic; Matheuristic; Neighborhood Search 1 Introduction In the Bin Packing Problem (BPP), there is a set of items and an unlimited set of bins of equal capacity. The objective is to allocate the items in bins, respecting their capacities and minimizing the number of used bins. A generalization of BPP is the Colorful Bin Packing Problem (CBPP), where each item also has a color, and there is an additional constraint that two items of the same color cannot be packed side by side in the same bin. The CBPP was proposed independently by B¨ohm et al [5] and D´osa and Epstein [17], being studied in these articles only from the point of view of online algorithms and approximation algorithms. In particular, B¨ohm et al. show that the adaptations of the BPP classic algorithms First Fit, Best Fit, and Worst Fit for the CBPP have unbounded approximation ratios. Moreover, the authors prove that any online algorithm has an asymptotic competitive ratio of at least 2.5 and show an (absolute) 3.5-competitive algorithm. Also, some years before these works, Balogh et al [2] introduced the Black and White Bin Packing Problem (BWBPP), a particular case of CBPP with only two colors. Just like Balogh et al [2], the following works Chen et al [13], Balogh et al [3], and B¨ohm et al [6] studied approximation algorithms for online and offline versions of BWBPP and CBPP. An exact and linear algorithm for CBPP was proposed by Alsarhan et al [1] for the particular case where all items have zero weight. Later, Borges and Schouery [7], Borges et al [8] proposed four exact algorithms for the CBPP. The first two are pseudo-polynomial formulations based on Arc Flow Formulation for the BPP proposed by Carvalho [11]. On the other hand, the last two are set-partition formulations solved by the VRPSolver framework proposed by Pessoa et al [30]. Balogh et al [2] presents two applications for CBPP. The first one is in packing problems of twodimensional objects of similar widths that must be packed alternately. For example, consider the contents of information and advertisement on video-content sharing portals (like YouTube) and Social Media, which should fit under each other due to the tiny display of smartphones. Thus, the maximum content that fits in one screen is the bin capacity, the content height is the item size, the information contents have the color white, and advertisement contents have the color black. The second application is commercial breaks on radio or TV, where commercials of different lengths and genres must be scheduled for commercial blocks. In this case, the length of the commercial blocks represents the bin capacity, and the commercial duration is the item size. Two commercials of the same genre have the same color to prohibit them from being transmitted in sequence. The CBPP is an NP-hard problem since it is a generalization of BPP (Karp [25]). For dealing with NP-hard problems, in addition to the exact algorithm approach, another interesting approach is heuristic algorithms, in particular, the meta-heuristic approach, which commonly finds high-quality solutions in reasonable computational time. In this article, we study CBPP using meta-heuristic and matheuristic approaches, and, as far as we know, it is the first study in the literature to use this approach. Next, we present a literature review of heuristic approaches to BPP and present our contribution to CBPP. 1.1 Literature Review Many authors have studied BPP and its variants in the literature, using much as exact algorithms as meta-heuristic algorithms. Next, we present a review of the best-known algorithms for BPP and its generalization called the Cutting Stock Problem (CSP). Also, we introduce the main studies that utilize the VNS in the BPP. The BPP exact algorithms are based on two main formulation types: the formulation of Gilmore and Gomory [21] and the Arc Flow Formulation of Carvalho [11]. Other works that use Gilmore and Gomory Formulation are Vance [31], Belov and Scheithauer [4], and Wei et al [32]. On the other hand, for the arc flow, there are the following works Brand˜ao and Pedroso [9], Delorme and Iori [14], and de Lima et al [26]. Conversely, in the heuristic algorithm approach, Fleszar and Hindi [19] study the Variable Neighborhood Search (VNS) meta-heuristic applied to BPP. In this work, the authors use a variation of the heuristic called Minimum Bin Slack, proposed by Gupta and Ho [23] as an initial solution for VNS. This combination proved to be quite powerful and is able to find the optimal solution in 1329 of the 1370 instances evaluated. Sequentially, Loh et al [27] use the Weight Annealing meta-heuristic in the BPP, which finds optimal solutions in all instances used by Fleszar and Hindi. Castelli and Vanneschi [12] propose a BPP heuristic called BPHS, which is used as the initial solution of the VNS. Despite promising results, this combination has shown a lower performance than two previous studies. Finally, Gonz´alez-San-Mart´ın et al [22] presents a comparative study among nineteen heuristic algorithms proposed in the last two decades. The best algorithm in the study was the Consistent Neighborhood Search (CNS) introduced by Buljubaˇsi´c and Vasquez [10], which solved 99.81% of the studied instances. 1.2 Our Contributions In this article, we study the CBPP using five initial construction heuristics. The first two are adaptations of heuristics of the BPP called Best Fit Decreasing and Minimum Bin Slack. The last three are heuristics proposed by us called Good Ordering, Hard BFD, and Two-by-Two. In sequence, we present a meta-heuristic approach for CBPP using the VNS, which uses the previous heuristics to build the initial solution. Lastly, we propose a new matheuristic that mixes linear programming with the meta-heuristics VNS and Greedy Randomized Adaptive Search (GRASP). Our results show that our approaches can find near-optimal solutions for instances of up to 104 items in seconds. In particular, for all instances studied of up to 2 × 103 items and with the known optimal solution, our approaches, in 60 seconds, find solutions that use at most one extra bin than an optimal solution. The article is organized as follows. In Section 2, we present a formal definition for CBPP. Next, in Section 3, we present some convenient notations and prove a proposition that provides easy handling of color constraints. In Section 4, we propose an Auxiliary Algorithm, which we use as a subroutine in some of our algorithms. The initial heuristics, the VNS, and the matheuristic are introduced, respectively, in Sections 5, 6, and 7. Finally, the computational results and conclusion are present in Sections 8 and 9. This paper is available on arxiv under CC 4.0 license. Authors: (1) Renan F. F. da Silva, Institute of Computing, University of Campinas; (2) Yulle G. F. Borges, Institute of Computing, University of Campinas; (3) Rafael C. S. Schouery, Institute of Computing, University of Campinas. Authors: (1) Renan F. F. da Silva, Institute of Computing, University of Campinas; (2) Yulle G. F. Borges, Institute of Computing, University of Campinas; (3) Rafael C. S. Schouery, Institute of Computing, University of Campinas. Abstract The Colorful Bin Packing Problem (CBPP) is a generalization of the Bin Packing Problem (BPP). The CBPP consists of packing a set of items, each with a weight and a color, in bins of limited capacity, minimizing the number of used bins and satisfying the constraint that two items of the same color cannot be packed side by side in the same bin. In this article, we proposed an adaptation of BPP heuristics and new heuristics for the CBPP. Moreover, we propose a set of fast neighborhood search algorithms for CBPP. These neighborhoods are applied in a meta-heuristic approach based on the Variable Neighborhood Search (VNS) and a matheuristic approach that mixes linear programming with the meta-heuristics VNS and Greedy Randomized Adaptive Search (GRASP). The results indicate that our matheuristic is superior to VNS and that both approaches can find near-optimal solutions for a large number instances, even for instances with many items. Keywords: Cutting; Packing; Meta-heuristic; Matheuristic; Neighborhood Search Keywords: 1 Introduction In the Bin Packing Problem (BPP), there is a set of items and an unlimited set of bins of equal capacity. The objective is to allocate the items in bins, respecting their capacities and minimizing the number of used bins. A generalization of BPP is the Colorful Bin Packing Problem (CBPP), where each item also has a color, and there is an additional constraint that two items of the same color cannot be packed side by side in the same bin. The CBPP was proposed independently by B¨ohm et al [5] and D´osa and Epstein [17], being studied in these articles only from the point of view of online algorithms and approximation algorithms. In particular, B¨ohm et al. show that the adaptations of the BPP classic algorithms First Fit, Best Fit, and Worst Fit for the CBPP have unbounded approximation ratios. Moreover, the authors prove that any online algorithm has an asymptotic competitive ratio of at least 2.5 and show an (absolute) 3.5-competitive algorithm. Also, some years before these works, Balogh et al [2] introduced the Black and White Bin Packing Problem (BWBPP), a particular case of CBPP with only two colors. Just like Balogh et al [2], the following works Chen et al [13], Balogh et al [3], and B¨ohm et al [6] studied approximation algorithms for online and offline versions of BWBPP and CBPP. An exact and linear algorithm for CBPP was proposed by Alsarhan et al [1] for the particular case where all items have zero weight. Later, Borges and Schouery [7], Borges et al [8] proposed four exact algorithms for the CBPP. The first two are pseudo-polynomial formulations based on Arc Flow Formulation for the BPP proposed by Carvalho [11]. On the other hand, the last two are set-partition formulations solved by the VRPSolver framework proposed by Pessoa et al [30]. Balogh et al [2] presents two applications for CBPP. The first one is in packing problems of twodimensional objects of similar widths that must be packed alternately. For example, consider the contents of information and advertisement on video-content sharing portals (like YouTube) and Social Media, which should fit under each other due to the tiny display of smartphones. Thus, the maximum content that fits in one screen is the bin capacity, the content height is the item size, the information contents have the color white, and advertisement contents have the color black. The second application is commercial breaks on radio or TV, where commercials of different lengths and genres must be scheduled for commercial blocks. In this case, the length of the commercial blocks represents the bin capacity, and the commercial duration is the item size. Two commercials of the same genre have the same color to prohibit them from being transmitted in sequence. The CBPP is an NP-hard problem since it is a generalization of BPP (Karp [25]). For dealing with NP-hard problems, in addition to the exact algorithm approach, another interesting approach is heuristic algorithms, in particular, the meta-heuristic approach, which commonly finds high-quality solutions in reasonable computational time. In this article, we study CBPP using meta-heuristic and matheuristic approaches, and, as far as we know, it is the first study in the literature to use this approach. Next, we present a literature review of heuristic approaches to BPP and present our contribution to CBPP. 1.1 Literature Review Many authors have studied BPP and its variants in the literature, using much as exact algorithms as meta-heuristic algorithms. Next, we present a review of the best-known algorithms for BPP and its generalization called the Cutting Stock Problem (CSP). Also, we introduce the main studies that utilize the VNS in the BPP. The BPP exact algorithms are based on two main formulation types: the formulation of Gilmore and Gomory [21] and the Arc Flow Formulation of Carvalho [11]. Other works that use Gilmore and Gomory Formulation are Vance [31], Belov and Scheithauer [4], and Wei et al [32]. On the other hand, for the arc flow, there are the following works Brand˜ao and Pedroso [9], Delorme and Iori [14], and de Lima et al [26]. Conversely, in the heuristic algorithm approach, Fleszar and Hindi [19] study the Variable Neighborhood Search (VNS) meta-heuristic applied to BPP. In this work, the authors use a variation of the heuristic called Minimum Bin Slack, proposed by Gupta and Ho [23] as an initial solution for VNS. This combination proved to be quite powerful and is able to find the optimal solution in 1329 of the 1370 instances evaluated. Sequentially, Loh et al [27] use the Weight Annealing meta-heuristic in the BPP, which finds optimal solutions in all instances used by Fleszar and Hindi. Castelli and Vanneschi [12] propose a BPP heuristic called BPHS, which is used as the initial solution of the VNS. Despite promising results, this combination has shown a lower performance than two previous studies. Finally, Gonz´alez-San-Mart´ın et al [22] presents a comparative study among nineteen heuristic algorithms proposed in the last two decades. The best algorithm in the study was the Consistent Neighborhood Search (CNS) introduced by Buljubaˇsi´c and Vasquez [10], which solved 99.81% of the studied instances. 1.2 Our Contributions In this article, we study the CBPP using five initial construction heuristics. The first two are adaptations of heuristics of the BPP called Best Fit Decreasing and Minimum Bin Slack. The last three are heuristics proposed by us called Good Ordering, Hard BFD, and Two-by-Two. In sequence, we present a meta-heuristic approach for CBPP using the VNS, which uses the previous heuristics to build the initial solution. Lastly, we propose a new matheuristic that mixes linear programming with the meta-heuristics VNS and Greedy Randomized Adaptive Search (GRASP). Our results show that our approaches can find near-optimal solutions for instances of up to 104 items in seconds. In particular, for all instances studied of up to 2 × 103 items and with the known optimal solution, our approaches, in 60 seconds, find solutions that use at most one extra bin than an optimal solution. The article is organized as follows. In Section 2, we present a formal definition for CBPP. Next, in Section 3, we present some convenient notations and prove a proposition that provides easy handling of color constraints. In Section 4, we propose an Auxiliary Algorithm, which we use as a subroutine in some of our algorithms. The initial heuristics, the VNS, and the matheuristic are introduced, respectively, in Sections 5, 6, and 7. Finally, the computational results and conclusion are present in Sections 8 and 9. This paper is available on arxiv under CC 4.0 license. This paper is available on arxiv under CC 4.0 license. available on arxiv

Fast Neighborhood Search Heuristics for the Colorful Bin Packing Problem

Bitcoin Ordinals Explained: A Comprehensive Guide

This paper is available on arxiv under CC 4.0 license. Authors: (1) Qin Wang, CSIRO Data61, Australia; (2) Shiping Chen, CSIRO Data61, Australia. Table of Links Abstract & Introduction Ethereum Accounts Account Abstraction Security Analysis Further Discussions Conclusion & References IV. SECURITY ANALYSIS We explore the potential security issues that may exist in new account formats after its update. We recall a series of account threats and related smart contract risks and develop a framework to evaluate potential risks in account abstraction. Conventional analyses of Ethereum often partition it into four distinct layers (e.g., [20]): the application layer (including accounts, smart contracts, and EVM), the data layer (including transactions, blocks, and events), the consensus layer (involving PoX mechanisms and incentives), and the network layer (comprising node discovery, message propagation, and verification). Given the central theme of this endeavor around accounts, we deliberately narrow our purview to the application layer, exploring the potential vulnerabilities targeting accounts, smart contracts, and related solidity language. Additionally, our coverage extends to certain elements within the transaction and block layers. Focused scope. (Tab.III). Inspired by a series of elegant investigations [20]–[27], we distill a succinct overview of vulnerabilities present within the current Ethereum application layer. These vulnerabilities can be broadly attributed to three factors: design pitfalls, vulnerabilities intrinsic to the Solidity programming language, and errors stemming from contract programming. Within each of these categories, we identify and elaborate upon sub-vulnerabilities ( ) that manifest across diverse functions and processes ( ). Our discussion then delves into the potential of account abstraction to alleviate these vulnerabilities. We examine the feasibility of employing account abstraction to mitigate these issues and explore strategies for implementation that could effectively address these concerns ( ). Framework design Vulnerability CauseFmWhere byWhichFunc . Our discussions are categorized into two facets, contingent upon the effectiveness of AA updates. Analyses This vulnerability centers around a concurrency issue where the blockchain’s future state is contingent upon the sequence of transaction execution, a process influenced by miners who group transactions into blocks based on incentives. The design of AA effectively mitigates this problem by employing bundlers to manage and submit transactions. Unlike miners or validators, bundlers have a limited scope of impacting transaction orders, as they handle a relatively small number of transactions. ✓ Transaction disorder. This vulnerability happens when a contract utilizes block.timestamp in crucial operations or as a means of generating randomness. It can be exploited by a malicious miner who has the capability to manipulate the value of block.timestamp (evidenced by [28]). The introduction of account abstraction helps address this concern, as AA reduces reliance on block.timestamp and shifts focus to bundlers, which are better suited for managing transactions and ensuring more controlled and secure interactions. ✓ Timestamp manipulation. The vulnerability stems from the EVM’s absence of address validity checking. In the process of encoding during contract invocation, if the encoded arguments’ length is insufficient, the EVM compensates by adding extra zeros to ensure 32 bytes. The AA design, through the involvement of the bundler and EntryPoint contract, can mitigate this issue. The design can effectively check the length of Ethereum addresses via msg.data. ✓ Short address. This vulnerability revolves around the manipulation of seed (a form of randomness) by malicious miners in gambling and lottery contracts that use pseudorandom number generation. AA might not directly enhance the security of random seed creation. This is because the process of generating random seeds typically involves functions within the contract logic rather than being directly embedded in accounts, which is the primary focus of AA. ✗ Randomness reliance. Malicious actors leverage empty accounts (devoid of nonce, balance, code, and storage) to initiate a DoS attack. These empty accounts still necessitate tracking within the Ethereum state trie and result in extended transaction processing times. AA is not equipped to mitigate this vulnerability, as it currently lacks verification mechanisms for empty accounts. However, the issue has been addressed through the implementation of EIP-161. ✗ Empty account. Ethereum uses the gas mechanism to deter abuse of computing resources, but this vulnerability emerges when the gas cost for resource consumption is improperly set. The exploitation of under-priced opcodes will consume a disproportionate amount of computing resources. This issue is not specifically addressed by the design of account abstraction, as it pertains to the gas pricing and resource allocation mechanisms within Ethereum. ✗ Under-price opcodes. Solidity would trigger an exception and terminate the call due to its rigid limit of 1,024 frames in the EVM call stack. Malicious actors could recursively call a contract, causing it to reach the maximum stack depth and fail subsequent external calls. While account abstraction cannot inherently address this specific vulnerability, changes in the Ethereum protocol (e.g., EIP-150) can indirectly impact AA as well as other aspects of the Ethereum ecosystem. ✗ Stack limit. The vulnerability arises when a contract’s execution demands a greater amount of gas than the permitted block gas limit. This occurs due to the inclusion of unbounded operations, such as loops, within the contract. AA design offers effective solutions to counter such issues in two primary ways: Firstly, the paymaster contract defines a fee policy that establishes clear limitations on gas limits. This mechanism ensures that transactions adhere to predefined gas constraints, preventing situations where execution demands exceed the allowable limit. Secondly, after AA’s update, every loop in smart contracts is invoked by an internal account rather than an external account. This setup grants the contract owner the ability to terminate loops if necessary ✓ Unbounded operations. This vulnerability stems from the inconsistency between the two return values from different calling routes: (i) directly referencing the callee contract instance and (ii) using low-level methods like send and call. Route-(i) will return the call back to the caller while Route- (ii) returns false to the caller. The inconsistency in behavior persists in the current state of the Solidity compiler, lacking rectification. AA design does not inherently address this issue, but updates or changes in the Solidity compiler or EVM could potentially influence AA as well. ✗ Inconsistent call returns. The vulnerability arises from Solidity’s behavior regarding uninitialized compound local variables, which, when not explicitly initialized, leads to the reference defaulting to slot 0 and overwriting content from that point onwards. AA design does not address this particular languagelevel issue, but it’s fortunate that the vulnerability was rectified by updating the version of Solidity. ✗ Pointer unreset. ✗ Outdated compiler. This vulnerability surfaces when a contract is compiled using an outdated compiler that harbors bugs. While AA design may not directly address this concern, it aligns with the solution: opting for an up-to-date compiler can substantially reduce the associated risks. Keeping the compiler current is a key practice to ensure the solidity and security of the compiled contracts. The vulnerability stems from the misnaming of the constructor function, inadvertently allowing unauthorized parties to take ownership of the contract. While AA design cannot directly solve this issue, it is fortunate that the problem was mitigated through the introduction of the constructor keyword ✓ Unclear constructor name. A contract can invoke another contract’s function by directly referencing the callee contract’s instance. However, the verification process carried out by the Solidity compiler only assesses whether the invoked function has been declared, and it doesn’t extend to verifying child functions that the invoked function inherits or parent functions that trigger it. This loophole can lead to unintended function execution. AA design does not offer a solution for this vulnerability, as it stems from Solidity’s insufficient type system. In this context, AA’s mechanisms don’t directly address this issue. ✗ Type casts. The vulnerability arises when an external contract initiates a function within the caller contract before the latter’s ongoing execution concludes—essentially manifesting as a cyclic call. This exploitable invocation persists until the caller contract depletes its gas resources. The foundation of this vulnerability can be attributed to two pivotal factors: (i) a contract’s decision-making hinges on specific state variables, which necessitate updating prior to invoking another contract; and (ii) the absence of a predefined gas limit for the transition. The introduction of AA can offer a partial solution to this issue. AA’s advantages, including reduced gas consumption and the ability to control transactions precisely through the paymaster contract, contribute to its potential mitigation. Additionally, user operations are treated as distinct inputs through independent transactions, fostering an environment free from dependencies on other contracts. This segregation of operations enhances security by reducing the risk of vulnerability to such cyclic call-related attacks. ✓ Reentrancy. The vulnerability arises from a scenario in which users can deposit funds into their contract accounts but are unable to later access or spend those deposited funds. Two primary factors contribute to this vulnerability: (i) contracts not furnishing a function for the expenditure of funds, and (ii) the inadvertent or deliberate termination of the callee contract. Account abstraction can offer partial mitigation of this vulnerability due to its transaction batching made by bundlers. With each user-requested operation, such as depositing funds into a smart contract, the transaction bundler gains an enhanced capability to inspect the contract before sending tokens. This heightened level of examination surpasses that of a standard user. The transaction bundler possesses the potential to ensure that operations transpire within the confines of the originating contract’s security. ✓ Frozen ether. Contract upgrading can be approached through two methods: (i) Dividing a contract into a proxy contract (non-updatable) and a logic contract (updatable). (ii) Employing a registry contract to maintain a record of the updated contracts. While the design of account abstraction is separate from these complexities, its contract functionalities align with its principles of upgradability. ✗ Upgraded contract. EVM offers an opcode called delegatecall, allowing the bytecode of a callee contract to be inserted into the bytecode of the caller contract. The vulnerability arises due to the possibility that a callee contract can modify state variables within the caller contract. Account abstraction does not inherently provide a distinct method to mitigate this particular vulnerability. The effectiveness of AA’s security in mitigating the issue hinges on its inherent design and solution. Fortunately, addressing this can be achieved via a straightforward means by enhancing its declaration. ✗ Delegatecall injection. This vulnerability stems from the situation where a callee contract causes the execution of a caller contract to be reverted. The design of account abstraction does not offer a significantly improved mitigation for this issue, as it fundamentally stems from contract programming practices. Consequently, this vulnerability might still be pertinent even within AA’s contract functionalities. ✗ Unexpected revert. This vulnerability surfaces when a contract’s decision-making relies on the values of this.balance or address(this).balance. These values can be manipulated by an attacker to gain unauthorized access to funds. The design update introduced by AA can potentially influence this vulnerability. In AA, all account addresses are explicitly bound with balances, unlike the implicit nature of the former design. This alteration could impact the dynamics of this vulnerability. ✓ Manipulated balance. This vulnerability arises when the result of an arithmetic operation surpasses the range of a Solidity data type. Regrettably, neither the Solidity compiler nor the EVM incorporates mechanisms to identify integer overflow or underflow. Account abstraction design limitations prevent it from overcoming these challenges. However, a solution to mitigate this vulnerability is within reach through the utilization of the SafeMath library, which has already proven effective in handling such arithmetic issues. ✗ Integer overflow. The vulnerability surfaces when a sender channels funds to multiple recipients via a proxy contract, bypassing individual transactions. In this mechanism, the proxy contract evaluates the authenticity of digital signatures from senders. If these signatures lack essential data (such as nonce or proxy contract address), a malicious recipient can exploit the situation to replay the message repeatedly, facilitating the withdrawal of surplus payments. In AA’s design, the role of the bundler mirrors that of a proxy agent. The bundler manages the accumulation of multiple transactions, functioning as a trusted entity. This approach not only aligns with the principles of a proxy contract but also serves as a safeguard against replay attacks. Further, the adoption of multi-sig techniques in AA can also contribute to the mitigation of issues. ✓ Insufficient signature. When the owner of a contract or a third party utilizes the self-destruct method to terminate a contract, the contract’s bytecode and storage will be erased. The vulnerability emerges from inadequate authentication mechanisms embedded within the contract. AA can indeed mitigate this issue by introducing multi-factor authentication, a more robust approach. This mandates the approval of multiple parties before a suicide operation is executed. By implementing this safeguard, AA enhances security by fortifying the authentication process and reducing the likelihood of unauthorized termination. ✓ Self-destruct contract. The problem originates from the utilization of tx.origin, a global variable within Solidity that identifies the original EOA responsible for triggering the transaction. The vulnerability arises when a contract employs tx.origin for authorization, thereby exposing it to phishing attacks. AA addresses this concern through its integration of EOA with contract invocation. The integration is pivotal as AA replaces tx.origin with msg.sender in default authentication. This shift is now a necessary step to bolster security against vulnerabilities stemming from tx.origin usage. ✓ tx.origin. This security issue arises due to the absence of identity verification when a caller invokes a function to transfer Ether to a random address. As seen in the mitigation strategies employed for previous balance-related vulnerabilities, the authentication mechanisms and the role of the bundler within account abstraction can aid in alleviating this problem. This ensures that only authorized entities are permitted to initiate such actions, thwarting unauthorized withdrawals. ✓ Wrong payment. Transaction details within a blockchain are inherently public due to the nature of the technology. While designating a state variable as private can limit other contracts’ access, the value of such a variable can still be inferred from transaction data. It’s worth noting that AA design does not specifically address the intricacies of transaction confidentiality; however, it aligns with the overall progression of smart contract evolution. Mitigation strategies encompass the implementation of cryptographic techniques like commitment schemes [29] and zero-knowledge proofs [30]. Additionally, hardware-based solutions [31] could also be explored to enhance the security of sensitive data within an untrusted environment. ✗ Lack of confidentiality. This vulnerability stems from inaccurately defining the visibility of a function, where the default public visibility can be exploited by attackers to improperly access functions. AA design doesn’t directly address this issue but aligns with its solution. This vulnerability is mitigated by requiring explicit specification of function visibility, a measure AA’s design also adheres to. ✗ Erroneous visibility.

Stories, News & Mentions about UPBOUND

Upbound’s Stories on HackerNoon