Sonr is a blockchain identity platform which enables developers to seamless create secure, encrypted, and web-3 ready experiences

In this article, I will be examining Web 3.0 as it relates to decentralized peer-to-peer (p2p) file sharing. Decentralized architecture is relatively new and still evolving. The same words can mean different things to different groups, so I’ll start by outlining some general concepts and definitions so we’re on the same page semantically. Web 2.0 is the current iteration of the internet — it’s the framework that the vast majority of web-based applications and services use today, it’s what most users think of as the “internet” or the “web”. The term Web 2.0 was coined in the dot com era by and loosely defined an evolution from Web 1.0 in terms of innovations in mobile, social, and cloud computing. The idea is far more nuanced but can be encapsulated by two main concepts — ‘services, not packaged software’ and ‘software above the level of a single device’. Tim O’Reilly While Web 2.0 is still highly value-creative, especially in the enterprise, there are successful projects emerging from what is potentially the next internet paradigm, referred to today as Web 3.0. It is still nascent, but many believe it is the next evolution of the internet and will be the framework that defines the next era of software. Just like Web 2.0, the definition of is nuanced (still evolving, and is often defined differently by different people). For the purposes of this article, I will simply define Web 3.0 as a shift to decentralized networks that rely on a network of peers rather than a client-server network reliant on centralized infrastructure. We’ll get more into the details of p2p networks, but let’s first discuss why decentralization is valuable. Web 3.0 Decentralized networks are often more performant. Let’s look at a file sharing use-case, where user A wants to send a video file to user B. In a centralized client-server network, the video file is uploaded from user A to a server that is downloading, and then user B downloads the file (which the server is then uploading). The performance is limited by the upload speed of user A, the download speed of user B, the upload & download speed of the server, and the distance between each party. In a pure peer-to-peer system, the file is transported directly from one peer to another. It is only limited by the upload speed of user A, the download speed of user B, and the distance between them. Accordingly, if the users and the server are all close together and the server upload/download capacity is higher than users A and B, then performance may equal that of a p2p network. However, as the distance between the users and the server increases and/or the performance of the server decreases (e.g., from high demand), the client-server network will perform poorly compared to the p2p network. Decentralized networks are - they get stronger as additional ‘stress’ is added. As more users (peers) are added to the network, the network performance and availability actually increases. In Web 2.0 social applications, this is seen in ‘network effects’: the value of the network/product increases as the network size increases. antifragile Unfortunately, this is not the case for the underlying infrastructure. As an application based on a client-server model increases in popularity, the application provider must increase server capacity/central compute to maintain performance and availability. Decentralized networks are trustless - the participants involved do not need to know or trust each other or a 3rd party for the system to function. Due to the client-server relationship, Web 2.0 applications are reliant on centralized servers/services, which means users must inherently trust this central authority (person or entity that owns/operates the servers). This central authority has a unilateral ability to define the rules. In practice, this often manifests as a conflict around data ownership. For example, the popular storage and file sharing service dropbox uses a client-server model: users upload data to centralized servers owned and operated by dropbox, and then download data from these same servers. Accordingly, all data passes through dropbox, and dropbox has the technical ability to read & copy this data. Thus, the only thing protecting a dropbox user’s data is the user agreement that they trust dropbox to abide by (and not change). In a decentralized p2p network, there is no central authority. Users are the owners and operators of their data, and trust resides with the software itself (rather than the operators). Decentralized networks are more secure. It’s in the name: peer-to-peer. Data is directly uploaded from one peer and downloaded by the other without using a middle man (central server). This means that there is no central authority or custodian of your data (unless you choose so). Also, when there is a central authority, there is a higher incentive for attack since data/value is consolidated (i.e. an attacker derives lots of value from being successful once). Conversely, when data/value is highly distributed, an attacker would have to have exponentially more successful attempts to derive value of the same magnitude. Decentralization reduces the incentive/payout for attackers by orders of magnitude. Now that we have some shared context, let’s discuss decentralized p2p file sharing specifically. At this point, you might be thinking, “Ok, decentralization sounds great, but what’s new about this? Haven’t services, like p2p file sharing, existed for over a decade?” To some extent, you would be correct, but there are some essential differences between Web 2.0 p2p services and the ones we can build today on Web 3.0. Let’s take BitTorrent, which was initially released in 2001, as an example. BitTorrent has three main issues (which are solvable today): it is not fully decentralized, it lacks an incentive model, and the code is not usable or extensible. (and other similar services) is not decentralized in two important ways: it uses centralized servers to track peers and store content metadata. Although peer-to-peer connections are made (to improve performance, i.e., by increasing the number of ‘seeders’), the instantiation of these connections requires a special server called a tracker, which assists in the communication between peers. BitTorrent The tracker server also keeps track of where file copies reside on peer machines, which ones are available at time of the client request, and helps coordinate efficient transmission and reassembly of the copied file. The use of trackers increases the COGS of the service provider and limits the privacy/security of users. BitTorrent doesn’t have a proper incentive model. When a user begins using the service to download a file they are called a ‘leecher’, once they have pieces of the file (or if they have other files already stored that other users would like) they can become a ‘seeder’ that is also uploading files to other users. BitTorrent will prioritize downloads to seeding users (and users that are seeding at faster upload rates are further prioritized). However, other than this, there is no real incentive to seed or store files. Since BitTorrent is reliant on seeding and file availability, it is vulnerable without an incentive model. BitTorrent (and similar services) have that make it very difficult to leverage this previous work. Some examples include: Lack of good documentation (or none at all), restrictive licensing (or no licensing), no easy to reach point-of-contact, closed source (or the source doesn’t exist anymore), the implementation doesn’t expose a friendly API, these projects are tightly coupled with a specific use-case. classic software usability issues Ultimately, this means that the service’s ability to adapt and change with user needs and requirements is severely limited. In many cases makes it impossible to address new use-cases. Specifically, BitTorrent has made some improvements (it is less reliant on trackers than in the past), but it is essentially the same service today as it was 20 years ago. Let’s now examine Web 3.0 capabilities by continuing to use file sharing as an example. Today, you can build fully decentralized p2p applications. , which developed (for decentralized transport) and (for decentralized storage) is a leader in this space. Protocol Labs Libp2p IPFS Specifically, Libp2p’s transport protocol (via circuit relay & NAT traversal) solves connecting two peers without a special tracker server. Also, IPFS (a decentralized storage service that uses content addressing) allows for fully decentralized storage and is not reliant on either content servers or an extremely large number of live/available peers to achieve file availability. The Web 3.0 concept of crypto/tokens solves the incentive issue. Specifically, is a decentralized storage network that turns cloud storage into an algorithmic market. Filecoin The market runs on a blockchain with a native protocol token (FIL), which miners earn by providing storage to clients. Conversely, clients spend FIL (hiring miners) to store or distribute data. Applied to a service like BitTorrent, this means that seeders would be compensated in Filecoin for storing files and making them available for download, and leechers would spend Filecoin to access them. This incentive structure would increase the incentive to seed and store files, which increases availability and performance to users. Monetary incentives also create a pathway to instantiate legitimate business models. Suppose the file storer were able to make money. In that case, the legitimate owners of files (media corps) might be willing to participate in the network. As Spotify proved, users don’t actually care much about the ‘free’ aspect of torrents. They care about ease of discovery and distribution. Most importantly, entities like Protocol Labs are solving software usability issues. They are the creators of Libp2p, IPFS, and Filecoin. All three of these projects are open-source, have up-to-date documentation, and are supported (by a reachable team). Additionally, protocols like Libp2p are highly modular and built for general-purpose use. This means that products built with Libp2p are highly extensible and can change rapidly to address evolving user needs and new use-cases. In summary, decentralized p2p applications have many native benefits (especially in terms of performance and security), and now tools and infrastructure are in place to build robust p2p applications. This brings me to the reason I’m writing this article: the beta release of - a pure peer-to-peer (decentralized) file sharing application. Sonr Think of it as a cross-platform airdrop that works with any file size, under any network conditions, at any distance. It is built with Libp2p and may incorporate IPFS/Filecoin or other decentralized storage networks in the future. Our vision for Sonr is to create a universal and seamless data delivery protocol. Stay tuned for more news and updates about Sonr soon!

The Hitchhiker's Guide to Web 3.0

Sonr

Five Weeks in a Balloon by Jules Verne, is part of the HackerNoon Books Series. You can jump to any chapter in this book . Chapter XXXIX here CHAPTER THIRTY-NINTH. The Country in the Elbow of the Niger.—A Fantastic View of the Hombori Mountains.—Kabra.—Timbuctoo.—The Chart of Dr. Barth.—A Decaying City.—Whither Heaven wills. During this dull Monday, Dr. Ferguson diverted his thoughts by giving his companions a thousand details concerning the country they were crossing. The surface, which was quite flat, offered no impediment to their progress. The doctor’s sole anxiety arose from the obstinate northeast wind which continued to blow furiously, and bore them away from the latitude of Timbuctoo. The Niger, after running northward as far as that city, sweeps around, like an immense water-jet from some fountain, and falls into the Atlantic in a broad sheaf. In the elbow thus formed the country is of varied character, sometimes luxuriantly fertile, and sometimes extremely bare; fields of maize succeeded by wide spaces covered with broom-corn and uncultivated plains. All kinds of aquatic birds—pelicans, wild-duck, kingfishers, and the rest—were seen in numerous flocks hovering about the borders of the pools and torrents. From time to time there appeared an encampment of Touaregs, the men sheltered under their leather tents, while their women were busied with the domestic toil outside, milking their camels and smoking their huge-bowled pipes. By eight o’clock in the evening the had advanced more than two hundred miles to the westward, and our aëronauts became the spectators of a magnificent scene. Victoria A mass of moonbeams forcing their way through an opening in the clouds, and gliding between the long lines of falling rain, descended in a golden shower on the ridges of the Hombori Mountains. Nothing could be more weird than the appearance of these seemingly basaltic summits; they stood out in fantastic profile against the sombre sky, and the beholder might have fancied them to be the legendary ruins of some vast city of the middle ages, such as the icebergs of the polar seas sometimes mimic them in nights of gloom. “An admirable landscape for the ‘Mysteries of Udolpho’!” exclaimed the doctor. “Ann Radcliffe could not have depicted yon mountains in a more appalling aspect.” “Faith!” said Joe, “I wouldn’t like to be strolling alone in the evening through this country of ghosts. Do you see now, master, if it wasn’t so heavy, I’d like to carry that whole landscape home to Scotland! It would do for the borders of Loch Lomond, and tourists would rush there in crowds.” “Our balloon is hardly large enough to admit of that little experiment—but I think our direction is changing. Bravo!—the elves and fairies of the place are quite obliging. See, they’ve sent us a nice little southeast breeze, that will put us on the right track again.” In fact, the was resuming a more northerly route, and on the morning of the 20th she was passing over an inextricable network of channels, torrents, and streams, in fine, the whole complicated tangle of the Niger’s tributaries. Many of these channels, covered with a thick growth of herbage, resembled luxuriant meadow-lands. There the doctor recognized the route followed by the explorer Barth when he launched upon the river to descend to Timbuctoo. Eight hundred fathoms broad at this point, the Niger flowed between banks richly grown with cruciferous plants and tamarind-trees. Herds of agile gazelles were seen skipping about, their curling horns mingling with the tall herbage, within which the alligator, half concealed, lay silently in wait for them with watchful eyes. Victoria Long files of camels and asses laden with merchandise from Jenné were winding in under the noble trees. Ere long, an amphitheatre of low-built houses was discovered at a turn of the river, their roofs and terraces heaped up with hay and straw gathered from the neighboring districts. “There’s Kabra!” exclaimed the doctor, joyously; “there is the harbor of Timbuctoo, and the city is not five miles from here!” “Then, sir, you are satisfied?” half queried Joe. “Delighted, my boy!” “Very good; then every thing’s for the best!” In fact, about two o’clock, the Queen of the Desert, mysterious Timbuctoo, which once, like Athens and Rome, had her schools of learned men, and her professorships of philosophy, stretched away before the gaze of our travellers. Ferguson followed the most minute details upon the chart traced by Barth himself, and was enabled to recognize its perfect accuracy. The city forms an immense triangle marked out upon a vast plain of white sand, its acute angle directed toward the north and piercing a corner of the desert. In the environs there was almost nothing, hardly even a few grasses, with some dwarf mimosas and stunted bushes. As for the appearance of Timbuctoo, the reader has but to imagine a collection of billiard-balls and thimbles—such is the bird’s-eye view! The streets, which are quite narrow, are lined with houses only one story in height, built of bricks dried in the sun, and huts of straw and reeds, the former square, the latter conical. Upon the terraces were seen some of the male inhabitants, carelessly lounging at full length in flowing apparel of bright colors, and lance or musket in hand; but no women were visible at that hour of the day. “Yet they are said to be handsome,” remarked the doctor. “You see the three towers of the three mosques that are the only ones left standing of a great number—the city has indeed fallen from its ancient splendor! At the top of the triangle rises the Mosque of Sankore, with its ranges of galleries resting on arcades of sufficiently pure design. Farther on, and near to the Sane-Gungu quarter, is the Mosque of Sidi-Yahia and some two-story houses. But do not look for either palaces or monuments: the sheik is a mere son of traffic, and his royal palace is a counting-house.” “It seems to me that I can see half-ruined ramparts,” said Kennedy. “They were destroyed by the Fouillanes in 1826; the city was one-third larger then, for Timbuctoo, an object generally coveted by all the tribes, since the eleventh century, has belonged in succession to the Touaregs, the Sonrayans, the Morocco men, and the Fouillanes; and this great centre of civilization, where a sage like Ahmed-Baba owned, in the sixteenth century, a library of sixteen hundred manuscripts, is now nothing but a mere half-way house for the trade of Central Africa.” The city, indeed, seemed abandoned to supreme neglect; it betrayed that indifference which seems epidemic to cities that are passing away. Huge heaps of rubbish encumbered the suburbs, and, with the hill on which the market-place stood, formed the only inequalities of the ground. When the passed, there was some slight show of movement; drums were beaten; but the last learned man still lingering in the place had hardly time to notice the new phenomenon, for our travellers, driven onward by the wind of the desert, resumed the winding course of the river, and, ere long, Timbuctoo was nothing more than one of the fleeting reminiscences of their journey. Victoria “And now,” said the doctor, “Heaven may waft us whither it pleases!” “Provided only that we go westward,” added Kennedy. “Bah!” said Joe; “I wouldn’t be afraid if it was to go back to Zanzibar by the same road, or to cross the ocean to America.” “We would first have to be able to do that, Joe!” “And what’s wanting, doctor?” “Gas, my boy; the ascending force of the balloon is evidently growing weaker, and we shall need all our management to make it carry us to the sea-coast. I shall even have to throw over some ballast. We are too heavy.” “That’s what comes of doing nothing, doctor; when a man lies stretched out all day long in his hammock, he gets fat and heavy. It’s a lazybones trip, this of ours, master, and when we get back every body will find us big and stout.” “Just like Joe,” said Kennedy; “just the ideas for him: but wait a bit! Can you tell what we may have to go through yet? We are still far from the end of our trip. Where do you expect to strike the African coast, doctor?” “I should find it hard to answer you, Kennedy. We are at the mercy of very variable winds; but I should think myself fortunate were we to strike it between Sierra Leone and Portendick. There is a stretch of country in that quarter where we should meet with friends.” “And it would be a pleasure to press their hands; but, are we going in the desirable direction?” “Not any too well, Dick; not any too well! Look at the needle of the compass; we are bearing southward, and ascending the Niger toward its sources.” “A fine chance to discover them,” said Joe, “if they were not known already. Now, couldn’t we just find others for it, on a pinch?” “Not exactly, Joe; but don’t be alarmed: I hardly expect to go so far as that.” At nightfall the doctor threw out the last bags of sand. The rose higher, and the blow-pipe, although working at full blast, could scarcely keep her up. At that time she was sixty miles to the southward of Timbuctoo, and in the morning the aëronauts awoke over the banks of the Niger, not far from Lake Debo. Victoria About HackerNoon Book Series: We bring you the most important technical, scientific, and insightful public domain books. This book is part of the public domain. Jules Verne (2002). Five Weeks in a Balloon. Urbana, Illinois: Project Gutenberg. Retrieved October 2022 https://www.gutenberg.org/cache/epub/3526/pg3526-images.html This eBook is for the use of anyone anywhere at no cost and with almost no restrictions whatsoever. You may copy it, give it away or re-use it under the terms of the Project Gutenberg License included with this eBook or online at , located at . www.gutenberg.org https://www.gutenberg.org/policy/license.html

The Country in the Elbow of the Niger

Five Weeks in a Balloon by Jules Verne, is part of the HackerNoon Books Series. You can jump to any chapter in this book . Chapter XXXVIII here CHAPTER THIRTY-EIGHTH. A Rapid Passage.—Prudent Resolves.—Caravans in Sight.—Incessant Rains.—Goa.—The Niger.—Golberry, Geoffroy, and Gray.—Mungo Park.—Laing.—René Caillié.—Clapperton.—John and Richard Lander. The 17th of May passed tranquilly, without any remarkable incident; the desert gained upon them once more; a moderate wind bore the toward the southwest, and she never swerved to the right or to the left, but her shadow traced a perfectly straight line on the sand. Victoria Before starting, the doctor had prudently renewed his stock of water, having feared that he should not be able to touch ground in these regions, infested as they are by the Aouelim-Minian Touaregs. The plateau, at an elevation of eighteen hundred feet above the level of the sea, sloped down toward the south. Our travellers, having crossed the Aghades route at Murzouk—a route often pressed by the feet of camels—arrived that evening, in the sixteenth degree of north latitude, and four degrees fifty-five minutes east longitude, after having passed over one hundred and eighty miles of a long and monotonous day’s journey. During the day Joe dressed the last pieces of game, which had been only hastily prepared, and he served up for supper a mess of snipe, that were greatly relished. The wind continuing good, the doctor resolved to keep on during the night, the moon, still nearly at the full, illumining it with her radiance. The ascended to a height of five hundred feet, and, during her nocturnal trip of about sixty miles, the gentle slumbers of an infant would not have been disturbed by her motion. Victoria On Sunday morning, the direction of the wind again changed, and it bore to the northwestward. A few crows were seen sweeping through the air, and, off on the horizon, a flock of vultures which, fortunately, however, kept at a distance. The sight of these birds led Joe to compliment his master on the idea of having two balloons. “Where would we be,” said he, “with only one balloon? The second balloon is like the life-boat to a ship; in case of wreck we could always take to it and escape.” “You are right, friend Joe,” said the doctor, “only that my life-boat gives me some uneasiness. It is not so good as the main craft.” “What do you mean by that, doctor?” asked Kennedy. “I mean to say that the new is not so good as the old one. Whether it be that the stuff it is made of is too much worn, or that the heat of the spiral has melted the gutta-percha, I can observe a certain loss of gas. It don’t amount to much thus far, but still it is noticeable. We have a tendency to sink, and, in order to keep our elevation, I am compelled to give greater dilation to the hydrogen.” Victoria “The deuce!” exclaimed Kennedy with concern; “I see no remedy for that.” “There is none, Dick, and that is why we must hasten our progress, and even avoid night halts.” “Are we still far from the coast?” asked Joe. “Which coast, my boy? How are we to know whither chance will carry us? All that I can say is, that Timbuctoo is still about four hundred miles to the westward. “And how long will it take us to get there?” “Should the wind not carry us too far out of the way, I hope to reach that city by Tuesday evening.” “Then,” remarked Joe, pointing to a long file of animals and men winding across the open desert, “we shall arrive there sooner than that caravan.” Ferguson and Kennedy leaned over and saw an immense cavalcade. There were at least one hundred and fifty camels of the kind that, for twelve of gold, or about twenty-five dollars, go from Timbuctoo to Tafilet with a load of five hundred pounds upon their backs. Each animal had dangling to its tail a bag to receive its excrement, the only fuel on which the caravans can depend when crossing the desert. mutkals These Touareg camels are of the very best race. They can go from three to seven days without drinking, and for two without eating. Their speed surpasses that of the horse, and they obey with intelligence the voice of the , or guide of the caravan. They are known in the country under the name of . khabir mehari Such were the details given by the doctor while his companions continued to gaze upon that multitude of men, women, and children, advancing on foot and with difficulty over a waste of sand half in motion, and scarcely kept in its place by scanty nettles, withered grass, and stunted bushes that grew upon it. The wind obliterated the marks of their feet almost instantly. Joe inquired how the Arabs managed to guide themselves across the desert, and come to the few wells scattered far between throughout this vast solitude. “The Arabs,” replied Dr. Ferguson, “are endowed by nature with a wonderful instinct in finding their way. Where a European would be at a loss, they never hesitate for a moment. An insignificant fragment of rock, a pebble, a tuft of grass, a different shade of color in the sand, suffice to guide them with accuracy. During the night they go by the polar star. They never travel more than two miles per hour, and always rest during the noonday heat. You may judge from that how long it takes them to cross Sahara, a desert more than nine hundred miles in breadth.” But the had already disappeared from the astonished gaze of the Arabs, who must have envied her rapidity. That evening she passed two degrees twenty minutes east longitude, and during the night left another degree behind her. Victoria On Monday the weather changed completely. Rain began to fall with extreme violence, and not only had the balloon to resist the power of this deluge, but also the increase of weight which it caused by wetting the whole machine, car and all. This continuous shower accounted for the swamps and marshes that formed the sole surface of the country. Vegetation reappeared, however, along with the mimosas, the baobabs, and the tamarind-trees. Such was the Sonray country, with its villages topped with roofs turned over like Armenian caps. There were few mountains, and only such hills as were enough to form the ravines and pools where the and snipes went sailing and diving through. Here and there, an impetuous torrent cut the roads, and had to be crossed by the natives on long vines stretched from tree to tree. The forests gave place to jungles, which alligators, hippopotami, and the rhinoceros, made their haunts. pintadoes “It will not be long before we see the Niger,” said the doctor. “The face of the country always changes in the vicinity of large rivers. These moving highways, as they are sometimes correctly called, have first brought vegetation with them, as they will at last bring civilization. Thus, in its course of twenty-five hundred miles, the Niger has scattered along its banks the most important cities of Africa.” “By-the-way,” put in Joe, “that reminds me of what was said by an admirer of the goodness of Providence, who praised the foresight with which it had generally caused rivers to flow close to large cities!” At noon the was passing over a petty town, a mere assemblage of miserable huts, which once was Goa, a great capital. Victoria “It was there,” said the doctor, “that Barth crossed the Niger, on his return from Timbuctoo. This is the river so famous in antiquity, the rival of the Nile, to which pagan superstition ascribed a celestial origin. Like the Nile, it has engaged the attention of geographers in all ages; and like it, also, its exploration has cost the lives of many victims; yes, even more of them than perished on account of the other.” The Niger flowed broadly between its banks, and its waters rolled southward with some violence of current; but our travellers, borne swiftly by as they were, could scarcely catch a glimpse of its curious outline. “I wanted to talk to you about this river,” said Dr. Ferguson, “and it is already far from us. Under the names of Dhiouleba, Mayo, Egghirreou, Quorra, and other titles besides, it traverses an immense extent of country, and almost competes in length with the Nile. These appellations signify simply ‘the River,’ according to the dialects of the countries through which it passes.” “Did Dr. Barth follow this route?” asked Kennedy. “No, Dick: in quitting Lake Tchad, he passed through the different towns of Bornou, and intersected the Niger at Say, four degrees below Goa; then he penetrated to the bosom of those unexplored countries which the Niger embraces in its elbow; and, after eight months of fresh fatigues, he arrived at Timbuctoo; all of which we may do in about three days with as swift a wind as this.” “Have the sources of the Niger been discovered?” asked Joe. “Long since,” replied the doctor. “The exploration of the Niger and its tributaries was the object of several expeditions, the principal of which I shall mention: Between 1749 and 1758, Adamson made a reconnoissance of the river, and visited Gorea; from 1785 to 1788, Golberry and Geoffroy travelled across the deserts of Senegambia, and ascended as far as the country of the Moors, who assassinated Saugnier, Brisson, Adam, Riley, Cochelet, and so many other unfortunate men. Then came the illustrious Mungo Park, the friend of Sir Walter Scott, and, like him, a Scotchman by birth. Sent out in 1795 by the African Society of London, he got as far as Bambarra, saw the Niger, travelled five hundred miles with a slave-merchant, reconnoitred the Gambia River, and returned to England in 1797. He again set out, on the 30th of January, 1805, with his brother-in-law Anderson, Scott, the designer, and a gang of workmen; he reached Gorea, there added a detachment of thirty-five soldiers to his party, and saw the Niger again on the 19th of August. But, by that time, in consequence of fatigue, privations, ill-usage, the inclemencies of the weather, and the unhealthiness of the country, only eleven persons remained alive of the forty Europeans in the party. On the 16th of November, the last letters from Mungo Park reached his wife; and, a year later a trader from that country gave information that, having got as far as Boussa, on the Niger, on the 23d of December, the unfortunate traveller’s boat was upset by the cataracts in that part of the river, and he was murdered by the natives.” “And his dreadful fate did not check the efforts of others to explore that river?” “On the contrary, Dick. Since then, there were two objects in view: namely, to recover the lost man’s papers, as well as to pursue the exploration. In 1816, an expedition was organized, in which Major Grey took part. It arrived in Senegal, penetrated to the Fonta-Jallon, visited the Foullah and Mandingo populations, and returned to England without further results. In 1822, Major Laing explored all the western part of Africa near to the British possessions; and he it was who got so far as the sources of the Niger; and, according to his documents, the spring in which that immense river takes its rise is not two feet broad. “Easy to jump over,” said Joe. “How’s that? Easy you think, eh?” retorted the doctor. “If we are to believe tradition, whoever attempts to pass that spring, by leaping over it, is immediately swallowed up; and whoever tries to draw water from it, feels himself repulsed by an invisible hand.” “I suppose a man has a right not to believe a word of that!” persisted Joe. “Oh, by all means!—Five years later, it was Major Laing’s destiny to force his way across the desert of Sahara, penetrate to Timbuctoo, and perish a few miles above it, by strangling, at the hands of the Ouelad-shiman, who wanted to compel him to turn Mussulman.” “Still another victim!” said the sportsman. “It was then that a brave young man, with his own feeble resources, undertook and accomplished the most astonishing of modern journeys—I mean the Frenchman René Caillié, who, after sundry attempts in 1819 and 1824, set out again on the 19th of April, 1827, from Rio Nunez. On the 3d of August he arrived at Timé, so thoroughly exhausted and ill that he could not resume his journey until six months later, in January, 1828. He then joined a caravan, and, protected by his Oriental dress, reached the Niger on the 10th of March, penetrated to the city of Jenné, embarked on the river, and descended it, as far as Timbuctoo, where he arrived on the 30th of April. In 1760, another Frenchman, Imbert by name, and, in 1810, an Englishman, Robert Adams, had seen this curious place; but René Caillié was to be the first European who could bring back any authentic data concerning it. On the 4th of May he quitted this ‘Queen of the desert;’ on the 9th, he surveyed the very spot where Major Laing had been murdered; on the 19th, he arrived at El-Arouan, and left that commercial town to brave a thousand dangers in crossing the vast solitudes comprised between the Soudan and the northern regions of Africa. At length he entered Tangiers, and on the 28th of September sailed for Toulon. In nineteen months, notwithstanding one hundred and eighty days’ sickness, he had traversed Africa from west to north. Ah! had Callié been born in England, he would have been honored as the most intrepid traveller of modern times, as was the case with Mungo Park. But in France he was not appreciated according to his worth.” “He was a sturdy fellow!” said Kennedy, “but what became of him?” “He died at the age of thirty-nine, from the consequences of his long fatigues. They thought they had done enough in decreeing him the prize of the Geographical Society in 1828; the highest honors would have been paid to him in England. “While he was accomplishing this remarkable journey, an Englishman had conceived a similar enterprise and was trying to push it through with equal courage, if not with equal good fortune. This was Captain Clapperton, the companion of Denham. In 1829 he reentered Africa by the western coast of the Gulf of Benin; he then followed in the track of Mungo Park and of Laing, recovered at Boussa the documents relative to the death of the former, and arrived on the 20th of August at Sackatoo, where he was seized and held as a prisoner, until he expired in the arms of his faithful attendant Richard Lander.” “And what became of this Lander?” asked Joe, deeply interested. “He succeeded in regaining the coast and returned to London, bringing with him the captain’s papers, and an exact narrative of his own journey. He then offered his services to the government to complete the reconnoissance of the Niger. He took with him his brother John, the second child of a poor couple in Cornwall, and, together, these men, between 1829 and 1831, redescended the river from Boussa to its mouth, describing it village by village, mile by mile.” “So both the brothers escaped the common fate?” queried Kennedy. “Yes, on this expedition, at least; but in 1833 Richard undertook a third trip to the Niger, and perished by a bullet, near the mouth of the river. You see, then, my friends, that the country over which we are now passing has witnessed some noble instances of self-sacrifice which, unfortunately, have only too often had death for their reward.” About HackerNoon Book Series: We bring you the most important technical, scientific, and insightful public domain books. This book is part of the public domain. Jules Verne (2002). Five Weeks in a Balloon. Urbana, Illinois: Project Gutenberg. Retrieved October 2022 https://www.gutenberg.org/cache/epub/3526/pg3526-images.html This eBook is for the use of anyone anywhere at no cost and with almost no restrictions whatsoever. You may copy it, give it away or re-use it under the terms of the Project Gutenberg License included with this eBook or online at , located at . www.gutenberg.org https://www.gutenberg.org/policy/license.html

A Rapid Passage

Getting Started With Linux: Your Gateway to the Open-Source Realm

When we first came up with our “big idea” back in 2017, my partner Mike Wagner and I didn’t know how much impact it would have. The idea was simple enough: We wanted to develop a virtual appliance that could discover, provision, and maintain servers from any manufacturer. By 2020, that vision became a reality with the launch of our flagship platform, which we built entirely on open-source technologies. That strategy was working well from the start. Well enough, I should say. We knew we could improve our metal-as-a-service software appliance if we could just find a way to boost performance, ensure higher system uptime, and streamline the architecture. The only way to do that, we realized, was to replace the Linux operating system and KVM virtual machine solution with something that would give us those benefits. What we did know was that we wanted to continue with our strategy of utilizing open source—and ideally tied to a very permissive license and open governance model. That would give us more flexibility as our roadmap unfolded and enable extensive use and modification without stringent (or murky) conditions. Taking the plunge After our due diligence, we transitioned to the Unix-like open source FreeBSD operating system, along with the BSD hypervisor Bhyve. We liked that the FreeBSD project had off-the-shelf capabilities within a single distribution, and had plenty of mission-critical enterprise use cases (Netflix and Sony being two of the most well-known). Together with existing technologies like the Redfish standard, the new and improved platform we’d built (called Mojo) was ready to really take off. With FreeBSD and bhyve fully in place, we were now able to add edge data center provisioning to the platform, which helps discover, configure, and provision bare metal servers and storage within a data center and at the edge. It works really well with another new technology (PhotoISP, which I’ll discuss in a moment) to manage geographically distributed bare metal assets. These open-source changes to our infrastructure proved to be the silver bullet we were looking for. More organizations began to contact us, and we started landing some really interesting use cases, including Major League Baseball. Before the 2021 season, MLB was looking for a way to move from VMware to a containerized environment across 30 data centers. In a nutshell, they needed a way to quickly process 15 different input sources to track all the player and team statistics generated during a game. By more quickly provisioning servers, that data could be quickly gathered at the edge and uploaded into Google Cloud. The Mojo platform, backed by an evolved open-source strategy, helped streamline MLB’s deployment pipelines (cutting deployment time in half). It also simplified the provisioning and management of MLB’s bare metal edge sites and made it really easy to provision Anthos, BIOS, firmware, OS patches, and upgrades. They also saw big performance gains for their GPU-enabled apps and more optimized CI/CD pipelines. Expanding a vision backed by open source Albert Einstein once said that necessity is the mother of all invention. My family lives in the Blue Ridge Mountains, and when COVID hit, word spread quickly that several teachers and students were unable to stream their classes because they had insufficient rural broadband access. Mike and I brainstormed about how we could help, and we immediately recognized how Mojo, paired with a reliable and secure CPE, could solve the problem. That led development of the Photon ISP platform, which delivers routing, firewall, and QoS through the open-source FreeBSD operating system. It also monitors IDS/IPS for wireless and fixed fiber networks and ensures uninterrupted 4G/5G and Starlink failover as a hedge to wireless ISP services. Today, the Photon ISP platform has 50 rural customers, and that number continues to grow. And its uptime is unparalleled; we have FreeBSD CPEs that have been running for more than a year without issue. Speeds have been very consistent, streaming content has been reliable, and there has been almost zero latency. We’re still pushing the limits of what we can do with our optimized open-source infrastructure. Next, we plan to finish moving the remainder of the stack to FreeBSD once a viable OCI-compliant stack becomes available. In the meantime, we still use Linux within Bhyve virtual machines for a few of our services. Simply put, the strategic shift to open-source FreeBSD and Bhyve for our platforms has catalyzed innovation and customer growth. The success with MLB—cutting deployment times and improving performance—is a great example of how the right open-source pieces can have an outsized impact while being far more flexible (not to mention cost-efficient!) than proprietary alternatives. When we first came up with our “big idea” back in 2017, my partner Mike Wagner and I didn’t know how much impact it would have. The idea was simple enough: We wanted to develop a virtual appliance that could discover, provision, and maintain servers from any manufacturer. By 2020, that vision became a reality with the launch of our flagship platform, which we built entirely on open-source technologies. That strategy was working well from the start. Well enough , I should say. enough We knew we could improve our metal-as-a-service software appliance if we could just find a way to boost performance, ensure higher system uptime, and streamline the architecture. The only way to do that, we realized, was to replace the Linux operating system and KVM virtual machine solution with something that would give us those benefits. What we did know was that we wanted to continue with our strategy of utilizing open source—and ideally tied to a very permissive license and open governance model. That would give us more flexibility as our roadmap unfolded and enable extensive use and modification without stringent (or murky) conditions. Taking the plunge After our due diligence, we transitioned to the Unix-like open source FreeBSD operating system, along with the BSD hypervisor Bhyve . We liked that the FreeBSD project had off-the-shelf capabilities within a single distribution, and had plenty of mission-critical enterprise use cases (Netflix and Sony being two of the most well-known). Together with existing technologies like the Redfish standard, the new and improved platform we’d built (called Mojo) was ready to really take off. open source FreeBSD Bhyve Redfish With FreeBSD and bhyve fully in place, we were now able to add edge data center provisioning to the platform, which helps discover, configure, and provision bare metal servers and storage within a data center and at the edge. It works really well with another new technology (PhotoISP, which I’ll discuss in a moment) to manage geographically distributed bare metal assets. These open-source changes to our infrastructure proved to be the silver bullet we were looking for. More organizations began to contact us, and we started landing some really interesting use cases, including Major League Baseball. Before the 2021 season, MLB was looking for a way to move from VMware to a containerized environment across 30 data centers. In a nutshell, they needed a way to quickly process 15 different input sources to track all the player and team statistics generated during a game. By more quickly provisioning servers, that data could be quickly gathered at the edge and uploaded into Google Cloud. The Mojo platform, backed by an evolved open-source strategy, helped streamline MLB’s deployment pipelines (cutting deployment time in half). It also simplified the provisioning and management of MLB’s bare metal edge sites and made it really easy to provision Anthos, BIOS, firmware, OS patches, and upgrades. They also saw big performance gains for their GPU-enabled apps and more optimized CI/CD pipelines. Expanding a vision backed by open source Albert Einstein once said that necessity is the mother of all invention. My family lives in the Blue Ridge Mountains, and when COVID hit, word spread quickly that several teachers and students were unable to stream their classes because they had insufficient rural broadband access. Mike and I brainstormed about how we could help, and we immediately recognized how Mojo, paired with a reliable and secure CPE, could solve the problem. That led development of the Photon ISP platform, which delivers routing, firewall, and QoS through the open-source FreeBSD operating system. It also monitors IDS/IPS for wireless and fixed fiber networks and ensures uninterrupted 4G/5G and Starlink failover as a hedge to wireless ISP services. Today, the Photon ISP platform has 50 rural customers, and that number continues to grow. And its uptime is unparalleled; we have FreeBSD CPEs that have been running for more than a year without issue. Speeds have been very consistent, streaming content has been reliable, and there has been almost zero latency. We’re still pushing the limits of what we can do with our optimized open-source infrastructure. Next, we plan to finish moving the remainder of the stack to FreeBSD once a viable OCI-compliant stack becomes available. In the meantime, we still use Linux within Bhyve virtual machines for a few of our services. Simply put, the strategic shift to open-source FreeBSD and Bhyve for our platforms has catalyzed innovation and customer growth. The success with MLB—cutting deployment times and improving performance—is a great example of how the right open-source pieces can have an outsized impact while being far more flexible (not to mention cost-efficient!) than proprietary alternatives.

How Our Open Source OS Pivot Accelerated Product Innovation

AI in web3 security is neither a passing fad nor a makeshift solution. On the contrary, it is now a focal point in the fight against cryptocurrency thefts and cyber attacks. AI companies in web3 are training AI algorithms to detect potential cyber attacks on DeFi protocols in a bid to reduce the probability of an attack significantly while increasing the response time. The ability of AI algorithms to detect potential threats can save millions of dollars of investors’ funds, highlighting not just the growing importance of AI but also cementing its place in web3 security, which is often described as a $4 billion problem. Although the use of AI in crime detection has long been around, it has only found relevance in crypto security where cyber thefts are rampant. AI algorithms are trained through a process called machine learning to spot anomalies and patterns commonly associated with cyber thefts and other types of crypto-related cyber attacks such as phishing and ransomware attacks. AI’s ability to provide a sophisticated security system gives it an edge over malicious software used by bad actors in attacking DeFi protocols. The advancement that AI in web3 security represents is preeminently self-evident in the latest Sonne Finance hack. There are numerous examples of such exploits and AI use cases but this incident being the latest in a long list of cyberattacks on DeFi protocols deserves to be critically studied. This sentiment is also informed by the volume of media coverage it has generated in the last 24 hours. The Sonne Finance Hack Sonne Finance, a decentralized lending protocol, reportedly lost $20 million in a major hack on May 14, 2024. The attack occurred in seven different transactions and lasted for thirty minutes before Sonne Finance halted operations to avert further thefts. Four minutes before the attack, Cyvers AI, a web3 security firm, detected the potential threat and alerted Sonne Finance at about 10:30 pm to the imminent danger. Commenting on the Sonne Finance hack,  Meir Dolev Co-Founder and CTO of Cyvers.AI said: " The hacker exploited Compound V2 empty market vulnerability where he conducted a precision loss attack that led to more than $20 million loss in 7 different transactions. Cyvers have detected the attack four minutes before the first exploit transaction" At the initial stage, Cyvers detected an attack on SonneFinance's USD Coin (USDC) and Wrapped Ether (WETH) contracts, which had already lost $3 million, but the situation quickly escalated resulting in a total loss of over $20 million by the protocol. This incident underscores the growing importance of Artificial Intelligence (AI) in the detection and mitigation of cryptocurrency theft. Moreover, as traditional security methods often struggle to keep pace with the ever-evolving tactics of cybercriminals. AI, however, offers several advantages: Real-time monitoring: AI systems can be deployed to continuously analyze vast amounts of data, identifying suspicious patterns that might escape human analysts. Faster response times: AI systems can react to threats much quicker than humans, allowing for a timely response to mitigate damage. Improved threat detection: AI algorithms are constantly learning to detect anomalies and patterns and may soon become adept at identifying sophisticated hacking techniques. The Implications of The Sonne Finance Hack The Sonne Finance hack is yet another intriguing case of cryptocurrency theft recorded so far in 2024. It highlights the growing tendency of digital currencies to become mainstream and the security challenges surrounding them. While appropriate measures have been taken to avert further theft, the incident raises critical questions about the vulnerability of DeFi protocols. Here's a deeper look at some of them: DeFi's Security Flaws:Decentralized Finance has a massive potential to revolutionize the financial landscape but often lacks the robust security infrastructure of traditional financial institutions, exposing DeFi protocols to cybersecurity risks.
A Fast-Paced Landscape:Crypto attackers are constantly developing new methods to exploit vulnerabilities. AI's ability to adapt and learn from these evolving threats can provide an edge for DeFi protocols.
Proactive Security:The Sonne Finance incident highlights the importance of proactive security measures. Implementing AI-powered security systems before an attack occurs can significantly mitigate the risk of financial losses.
Increased Adoption of AI Security:This incident is likely to accelerate the adoption of AI security solutions within the cryptocurrency industry. Web3 companies may increasingly resort to  AI to protect their assets and user funds.
Increased Collaboration: The complex nature of cyber threats necessitates collaboration between security firms, DeFi protocols, and regulatory bodies. Sharing information and best practices can create a more robust defense system. Conclusion The cyberattack on Sonne Finance can be described as the doors of the temple of Janus. While the incident itself is a cause for concern, it also highlights the immense potential of AI in detecting potential cybersecurity threats and mitigating them. By implementing robust AI security systems and fostering collaboration across the industry, DeFi platforms can build a stronger defense against cyber threats and ensure a safer future for investors and users alike. AI in web3 security is neither a passing fad nor a makeshift solution. On the contrary, it is now a focal point in the fight against cryptocurrency thefts and cyber attacks. AI companies in web3 are training AI algorithms to detect potential cyber attacks on DeFi protocols in a bid to reduce the probability of an attack significantly while increasing the response time. The ability of AI algorithms to detect potential threats can save millions of dollars of investors’ funds, highlighting not just the growing importance of AI but also cementing its place in web3 security, which is often described as a $4 billion problem. Although the use of AI in crime detection has long been around, it has only found relevance in crypto security where cyber thefts are rampant. AI algorithms are trained through a process called machine learning to spot anomalies and patterns commonly associated with cyber thefts and other types of crypto-related cyber attacks such as phishing and ransomware attacks. AI’s ability to provide a sophisticated security system gives it an edge over malicious software used by bad actors in attacking DeFi protocols. The advancement that AI in web3 security represents is preeminently self-evident in the latest Sonne Finance hack. There are numerous examples of such exploits and AI use cases but this incident being the latest in a long list of cyberattacks on DeFi protocols deserves to be critically studied. This sentiment is also informed by the volume of media coverage it has generated in the last 24 hours. media coverage media coverage The Sonne Finance Hack Sonne Finance, a decentralized lending protocol, reportedly lost $20 million in a major hack on May 14, 2024. The attack occurred in seven different transactions and lasted for thirty minutes before Sonne Finance halted operations to avert further thefts. Four minutes before the attack, Cyvers AI, a web3 security firm, detected the potential threat and alerted Sonne Finance at about 10:30 pm to the imminent danger. Commenting on the Sonne Finance hack,  Meir Dolev Co-Founder and CTO of Cyvers.AI said: " The hacker exploited Compound V2 empty market vulnerability where he conducted a precision loss attack that led to more than $20 million loss in 7 different transactions. Cyvers have detected the attack four minutes before the first exploit transaction" " The hacker exploited Compound V2 empty market vulnerability where he conducted a precision loss attack that led to more than $20 million loss in 7 different transactions. Cyvers have detected the attack four minutes before the first exploit transaction" At the initial stage, Cyvers detected an attack on SonneFinance's USD Coin (USDC) and Wrapped Ether (WETH) contracts, which had already lost $3 million, but the situation quickly escalated resulting in a total loss of over $20 million by the protocol. This incident underscores the growing importance of Artificial Intelligence (AI) in the detection and mitigation of cryptocurrency theft. Moreover, as traditional security methods often struggle to keep pace with the ever-evolving tactics of cybercriminals. AI, however, offers several advantages: Real-time monitoring: AI systems can be deployed to continuously analyze vast amounts of data, identifying suspicious patterns that might escape human analysts. Real-time monitoring: AI systems can be deployed to continuously analyze vast amounts of data, identifying suspicious patterns that might escape human analysts. Faster response times: AI systems can react to threats much quicker than humans, allowing for a timely response to mitigate damage. Faster response times: AI systems can react to threats much quicker than humans, allowing for a timely response to mitigate damage. Improved threat detection: AI algorithms are constantly learning to detect anomalies and patterns and may soon become adept at identifying sophisticated hacking techniques. Improved threat detection: AI algorithms are constantly learning to detect anomalies and patterns and may soon become adept at identifying sophisticated hacking techniques. The Implications of The Sonne Finance Hack The Sonne Finance hack is yet another intriguing case of cryptocurrency theft recorded so far in 2024. It highlights the growing tendency of digital currencies to become mainstream and the security challenges surrounding them. While appropriate measures have been taken to avert further theft, the incident raises critical questions about the vulnerability of DeFi protocols. Here's a deeper look at some of them: DeFi's Security Flaws:Decentralized Finance has a massive potential to revolutionize the financial landscape but often lacks the robust security infrastructure of traditional financial institutions, exposing DeFi protocols to cybersecurity risks. A Fast-Paced Landscape:Crypto attackers are constantly developing new methods to exploit vulnerabilities. AI's ability to adapt and learn from these evolving threats can provide an edge for DeFi protocols. Proactive Security:The Sonne Finance incident highlights the importance of proactive security measures. Implementing AI-powered security systems before an attack occurs can significantly mitigate the risk of financial losses. Increased Adoption of AI Security:This incident is likely to accelerate the adoption of AI security solutions within the cryptocurrency industry. Web3 companies may increasingly resort to  AI to protect their assets and user funds. Increased Collaboration: The complex nature of cyber threats necessitates collaboration between security firms, DeFi protocols, and regulatory bodies. Sharing information and best practices can create a more robust defense system. DeFi's Security Flaws: Decentralized Finance has a massive potential to revolutionize the financial landscape but often lacks the robust security infrastructure of traditional financial institutions, exposing DeFi protocols to cybersecurity risks. DeFi's Security Flaws: A Fast-Paced Landscape: Crypto attackers are constantly developing new methods to exploit vulnerabilities. AI's ability to adapt and learn from these evolving threats can provide an edge for DeFi protocols. A Fast-Paced Landscape: Proactive Security: The Sonne Finance incident highlights the importance of proactive security measures. Implementing AI-powered security systems before an attack occurs can significantly mitigate the risk of financial losses. Proactive Security: Increased Adoption of AI Security: This incident is likely to accelerate the adoption of AI security solutions within the cryptocurrency industry. Web3 companies may increasingly resort to  AI to protect their assets and user funds. Increased Adoption of AI Security: Increased Collaboration: The complex nature of cyber threats necessitates collaboration between security firms, DeFi protocols, and regulatory bodies. Sharing information and best practices can create a more robust defense system. Increased Collaboration: Conclusion The cyberattack on Sonne Finance can be described as the doors of the temple of Janus. While the incident itself is a cause for concern, it also highlights the immense potential of AI in detecting potential cybersecurity threats and mitigating them. By implementing robust AI security systems and fostering collaboration across the industry, DeFi platforms can build a stronger defense against cyber threats and ensure a safer future for investors and users alike.

Cyvers Early Detection Of Sonne Finance Hack and The Importance Of AI In Web3 Security

Authors:
(1) Yagci, Nurce, HAW Hamburg, Germany & nurce.yagci@haw-hamburg.de;
(2) Sünkler, Sebastian, HAW Hamburg, Germany & sebastian.suenkler@haw-hamburg.de;
(3) Häußler, Helena, HAW Hamburg, Germany & helena.haeuessler@haw-hamburg.de;
(4) Lewandowski, Dirk, HAW Hamburg, Germany & dirk.lewandowski@haw-hamburg.de. Table of Links Abstract and Introduction Literature Review Objectives and Research Questions Methods Results Discussion Conclusion, Research Data, Acknowledgments, and References RESULTS Classification of domains Since we are working with sources from two different countries and the type of content behind some domains might not be immediately recognizable, we manually created a general set of categories based on the top 50 domains found across all results. Following this, we manually classified the 50 most popular domains for both the German and US results (see Table 1). When comparing the top 50 domains in the German results, News service dominates with 54%, while Movies & Entertainment and Sports make up 18% and 14%, respectively. In the US, 34% of the top 50 domains are Sports websites, with News services following close behind at 30%. Movies & Entertainment only make up 12%. These are the overall distributions of domain classes across all search engines; below, we look at the differences between the search engines. Variety of domains A comparison of the number of root domains in the search results of Google, Bing, DuckDuckGo, and Metager in Germany shows that Google has the greatest diversity by a small margin (see Figure 2). Overall, the values are very similar. However, it is noticeable that Google has the greatest variety of domains, especially in the first three positions. Interestingly, the greater diversity of Google's sources is even more pronounced in the US results. To examine the differences more closely, we look at the numbers below. Table 2 shows the cumulative frequencies for the German results. The number of unique root domains converges at the fifth position. We found 2,841 unique domains in Google's results, 2,783 unique domains for Bing, 2,707 unique domains for DuckDuckGo, and 2,683 unique domains for Metager in Germany. There are differences when comparing the search results in Germany and the US. It is notable that, in contrast to Germany, in the US results, there are only minor differences in the first positions. However, Google also shows the greatest variety of root domains in the search results in the US. This is more evident than in the German results (see Table 3). Overall, we found 4,085 unique domains in Google, 3,602 in Bing, 3,579 in DuckDuckGo, and 3,500 in Metager. Popular domains When comparing the top domains for each search engine across all German search results collected, we find a clear preference for Wikipedia in all of them (see Table 4). While Wikipedia is the most popular domain in all search engines, the frequency in the Google results is significantly lower, with only 658 compared to the number of occurrences in Bing being 1,948, in Metager 1,878, and in DuckDuckGo 1,752. The other domains in the top 10 across the search engines are News services (Google: 7, Bing: 4, DuckDuckGo: 3, Metager: 4). Sports make up 2 out of 10 for all but Google, with only one sports website. Surprisingly, Instagram is in the top 10 for DuckDuckGo and Metager, even though in Table 1, we showed that domains of the class Social Media only make up 4% of top domains in the German results. The same goes for Amazon, which is the second most frequent domain in DuckDuckGo, even though in the overall results, E-Commerce websites only make up 2%. Contrarily, in the US results, Wikipedia has the highest number of occurrences in Google with 1,892 compared to Bing's 1,388, Metager's 1,304, and DuckDuckGo's 1,287 (see Table 5). Still, what remains the same is that Wikipedia is the most popular domain across all search engines. Another difference in the US results is that Social Media sources are much more prevalent across all search engines, especially in Google, with Instagram as the second most frequent domain, Facebook as the third, and Twitter as the sixth. This is surprising when considering that Social Media domains only make up 6% of the top domains for the US results (see Table 1). Another finding is that YouTube, a subsidiary of Google, is the top 8th domain for all three alternatives but not for Google. Again, 30% of the top domains for Google, DuckDuckGo, and Metager are News services. For Bing, it is 40%. For each search engine, Sports websites make up 20% of domains. Exclusivity of domains This section compares the top 50 domains for each search engine to determine what source Google and its alternatives might exclusively offer to the user. Table 6 shows the domains found solely in Google's top 50 domains list for the German queries. Out of these 20 domains, eight are Sports websites, six are News services, and four belong to the Entertainment & Movies class, with one each for E-Commerce and Government websites. On the contrary, table 7 shows the list of domains found in the top 50 domains of all three alternatives but not in Google. These would be the domains missed by users who only use Google. Out of the 13 domains, four are News services, followed by three Movies & Entertainment and two Celebrities sources. Interestingly, Instagram and YouTube are missing from Google, as well. When implementing the same evaluation for the US results, we find that 17 domains are exclusively found in Google's top 50 domains, with five domains classified as Sports, four as Movies & Entertainment, another four as News services, three as Social Media and one as Celebrities (see Table 8). Notable is that social media giants Facebook and TikTok are only found in Google and not in the alternatives. On the opposite end, 13 domains are not found in the US results on Google (see Table 9). Six of them are News services, three are websites about Celebrities, two are about Movies & Entertainment, and one each are Sports and E-Commerce websites. Interestingly, Amazon is missing in Google's results, as well as msn.com, a service from Microsoft, like Bing itself. Distribution of domains Next, we look at the Gini coefficient to measure the concentration and statistical dispersion of root domains in search engines. The distributions across all search engines in both German and US results show very similar values. The results range from 0.77 to 0.79 and 0.73 to 0.76 for German and US results, respectively (see Figure 3). Overall, this means that there is an imbalance in the distribution of results from root domains and that some top sources dominate the search results for both countries. Similarity of domains Comparing the similarities of the top 10 German results, we find that Bing and Metager are most similar in terms of their top results, with 70%, followed by 64% overlap between DuckDuckGo and Metager and 63% between Bing and DuckDuckGo. However, when comparing the alternative search engines to Google, we find that the highest overlap is between Google and Bing with 28% and slightly lower ones with DuckDuckGo and Metager at 27%. Interestingly, the top 1 result overlap is higher between Google and Bing (31%) than the overlap between Bing and DuckDuckGo (15%). The results show a mean overlap of 30% of top 1, 40% of top 3, and 47% of top 10 results (table 10). This indicates that when looking at the entire first search results page, nearly half of the domains are the same in all search engines considered. For search engine users, this means that in addition to Google, this would also allow them to see search results that they would otherwise miss if they used the alternative search engine The same evaluation for the US yields similar results. Again, the overlap between Bing and Metager is the highest, but the margin between this pair and the other alternative pairings has been reduced. When looking at the top 10 results, Bing and Metager overlap by 65%, Bing and DuckDuckGo by 64%, and DuckDuckGo and Metager by 62%. While Google's overlap with the other search engines is lower in the US results, again, the pair of Google and Bing is slightly higher (25%) than the 24% overlap between Google and the other two alternatives (see Table 11). Narrowing the results down to the top 3, the overlap between Bing and DuckDuckGo increases minimally to 65% and remains the same for DuckDuckGo and Metager at 62%. Here, the overlap between Google and Bing is lower than in the German results, with only 26%. Furthermore, the top 1 result overlap between Google and Bing is at 29%, which is also lower than in the German results. Generally, compared to the German results, there is a higher overlap in the top 1 position for the pairs Google and DuckDuckGo (20%), Bing and DuckDuckGo (50%), and DuckDuckGo and Metager (47%). The averaged results make this trend clearer because the overlaps for the top 1, top 3, and top 10 results are similar, with 41%, 46%, and 44%, respectively. This paper is available on arxiv under CC 4.0 license. Authors: (1) Yagci, Nurce, HAW Hamburg, Germany & nurce.yagci@haw-hamburg.de; (2) Sünkler, Sebastian, HAW Hamburg, Germany & sebastian.suenkler@haw-hamburg.de; (3) Häußler, Helena, HAW Hamburg, Germany & helena.haeuessler@haw-hamburg.de; (4) Lewandowski, Dirk, HAW Hamburg, Germany & dirk.lewandowski@haw-hamburg.de. Authors: (1) Yagci, Nurce, HAW Hamburg, Germany & nurce.yagci@haw-hamburg.de; (2) Sünkler, Sebastian, HAW Hamburg, Germany & sebastian.suenkler@haw-hamburg.de; (3) Häußler, Helena, HAW Hamburg, Germany & helena.haeuessler@haw-hamburg.de; (4) Lewandowski, Dirk, HAW Hamburg, Germany & dirk.lewandowski@haw-hamburg.de. Table of Links Abstract and Introduction Abstract and Introduction Literature Review Literature Review Objectives and Research Questions Objectives and Research Questions Methods Methods Results Results Discussion Discussion Conclusion, Research Data, Acknowledgments, and References Conclusion, Research Data, Acknowledgments, and References RESULTS Classification of domains Since we are working with sources from two different countries and the type of content behind some domains might not be immediately recognizable, we manually created a general set of categories based on the top 50 domains found across all results. Following this, we manually classified the 50 most popular domains for both the German and US results (see Table 1). When comparing the top 50 domains in the German results, News service dominates with 54%, while Movies & Entertainment and Sports make up 18% and 14%, respectively. In the US, 34% of the top 50 domains are Sports websites, with News services following close behind at 30%. Movies & Entertainment only make up 12%. These are the overall distributions of domain classes across all search engines; below, we look at the differences between the search engines. Variety of domains A comparison of the number of root domains in the search results of Google, Bing, DuckDuckGo, and Metager in Germany shows that Google has the greatest diversity by a small margin (see Figure 2). Overall, the values are very similar. However, it is noticeable that Google has the greatest variety of domains, especially in the first three positions. Interestingly, the greater diversity of Google's sources is even more pronounced in the US results. To examine the differences more closely, we look at the numbers below. Table 2 shows the cumulative frequencies for the German results. The number of unique root domains converges at the fifth position. We found 2,841 unique domains in Google's results, 2,783 unique domains for Bing, 2,707 unique domains for DuckDuckGo, and 2,683 unique domains for Metager in Germany. There are differences when comparing the search results in Germany and the US. It is notable that, in contrast to Germany, in the US results, there are only minor differences in the first positions. However, Google also shows the greatest variety of root domains in the search results in the US. This is more evident than in the German results (see Table 3). Overall, we found 4,085 unique domains in Google, 3,602 in Bing, 3,579 in DuckDuckGo, and 3,500 in Metager. Popular domains When comparing the top domains for each search engine across all German search results collected, we find a clear preference for Wikipedia in all of them (see Table 4). While Wikipedia is the most popular domain in all search engines, the frequency in the Google results is significantly lower, with only 658 compared to the number of occurrences in Bing being 1,948, in Metager 1,878, and in DuckDuckGo 1,752. The other domains in the top 10 across the search engines are News services (Google: 7, Bing: 4, DuckDuckGo: 3, Metager: 4). Sports make up 2 out of 10 for all but Google, with only one sports website. Surprisingly, Instagram is in the top 10 for DuckDuckGo and Metager, even though in Table 1, we showed that domains of the class Social Media only make up 4% of top domains in the German results. The same goes for Amazon, which is the second most frequent domain in DuckDuckGo, even though in the overall results, E-Commerce websites only make up 2%. Contrarily, in the US results, Wikipedia has the highest number of occurrences in Google with 1,892 compared to Bing's 1,388, Metager's 1,304, and DuckDuckGo's 1,287 (see Table 5). Still, what remains the same is that Wikipedia is the most popular domain across all search engines. Another difference in the US results is that Social Media sources are much more prevalent across all search engines, especially in Google, with Instagram as the second most frequent domain, Facebook as the third, and Twitter as the sixth. This is surprising when considering that Social Media domains only make up 6% of the top domains for the US results (see Table 1). Another finding is that YouTube, a subsidiary of Google, is the top 8th domain for all three alternatives but not for Google. Again, 30% of the top domains for Google, DuckDuckGo, and Metager are News services. For Bing, it is 40%. For each search engine, Sports websites make up 20% of domains. Exclusivity of domains This section compares the top 50 domains for each search engine to determine what source Google and its alternatives might exclusively offer to the user. Table 6 shows the domains found solely in Google's top 50 domains list for the German queries. Out of these 20 domains, eight are Sports websites, six are News services, and four belong to the Entertainment & Movies class, with one each for E-Commerce and Government websites. On the contrary, table 7 shows the list of domains found in the top 50 domains of all three alternatives but not in Google. These would be the domains missed by users who only use Google. Out of the 13 domains, four are News services, followed by three Movies & Entertainment and two Celebrities sources. Interestingly, Instagram and YouTube are missing from Google, as well. When implementing the same evaluation for the US results, we find that 17 domains are exclusively found in Google's top 50 domains, with five domains classified as Sports, four as Movies & Entertainment, another four as News services, three as Social Media and one as Celebrities (see Table 8). Notable is that social media giants Facebook and TikTok are only found in Google and not in the alternatives. On the opposite end, 13 domains are not found in the US results on Google (see Table 9). Six of them are News services, three are websites about Celebrities, two are about Movies & Entertainment, and one each are Sports and E-Commerce websites. Interestingly, Amazon is missing in Google's results, as well as msn.com, a service from Microsoft, like Bing itself. Distribution of domains Next, we look at the Gini coefficient to measure the concentration and statistical dispersion of root domains in search engines. The distributions across all search engines in both German and US results show very similar values. The results range from 0.77 to 0.79 and 0.73 to 0.76 for German and US results, respectively (see Figure 3). Overall, this means that there is an imbalance in the distribution of results from root domains and that some top sources dominate the search results for both countries. Similarity of domains Comparing the similarities of the top 10 German results, we find that Bing and Metager are most similar in terms of their top results, with 70%, followed by 64% overlap between DuckDuckGo and Metager and 63% between Bing and DuckDuckGo. However, when comparing the alternative search engines to Google, we find that the highest overlap is between Google and Bing with 28% and slightly lower ones with DuckDuckGo and Metager at 27%. Interestingly, the top 1 result overlap is higher between Google and Bing (31%) than the overlap between Bing and DuckDuckGo (15%). The results show a mean overlap of 30% of top 1, 40% of top 3, and 47% of top 10 results (table 10). This indicates that when looking at the entire first search results page, nearly half of the domains are the same in all search engines considered. For search engine users, this means that in addition to Google, this would also allow them to see search results that they would otherwise miss if they used the alternative search engine The same evaluation for the US yields similar results. Again, the overlap between Bing and Metager is the highest, but the margin between this pair and the other alternative pairings has been reduced. When looking at the top 10 results, Bing and Metager overlap by 65%, Bing and DuckDuckGo by 64%, and DuckDuckGo and Metager by 62%. While Google's overlap with the other search engines is lower in the US results, again, the pair of Google and Bing is slightly higher (25%) than the 24% overlap between Google and the other two alternatives (see Table 11). Narrowing the results down to the top 3, the overlap between Bing and DuckDuckGo increases minimally to 65% and remains the same for DuckDuckGo and Metager at 62%. Here, the overlap between Google and Bing is lower than in the German results, with only 26%. Furthermore, the top 1 result overlap between Google and Bing is at 29%, which is also lower than in the German results. Generally, compared to the German results, there is a higher overlap in the top 1 position for the pairs Google and DuckDuckGo (20%), Bing and DuckDuckGo (50%), and DuckDuckGo and Metager (47%). The averaged results make this trend clearer because the overlaps for the top 1, top 3, and top 10 results are similar, with 41%, 46%, and 44%, respectively. This paper is available on arxiv under CC 4.0 license. This paper is available on arxiv under CC 4.0 license. available on arxiv

Domain Classification: Analyzing Source Types in Search Engine Results

Authors:
(1) Yagci, Nurce, HAW Hamburg, Germany & nurce.yagci@haw-hamburg.de;
(2) Sünkler, Sebastian, HAW Hamburg, Germany & sebastian.suenkler@haw-hamburg.de;
(3) Häußler, Helena, HAW Hamburg, Germany & helena.haeuessler@haw-hamburg.de;
(4) Lewandowski, Dirk, HAW Hamburg, Germany & dirk.lewandowski@haw-hamburg.de. Table of Links Abstract and Introduction Literature Review Objectives and Research Questions Methods Results Discussion Conclusion, Research Data, Acknowledgments, and References LITERATURE REVIEW Search result overlap By the mid-1990s, researching the overlap of search results between different search engines sparked interest for the purpose of estimating the size of the Web (Bharat, 1998; Ding & Marchionini, 1996; Lawrence & Giles, 1999). The generally small degrees of overlap indicated diverse underlying databases, each of limited size. Therefore, metasearch engines that combined the results of several search engines were meant to provide an additional value (Chignell et al., 1999) and attracted further research (i.e., Meng et al., 2002). Consequently, Spink et al. (Spink et al., 2006) included the meta search engine Dogpile in their extensive study on the overlap of search results. For a collection of 12,570 queries, the results found on the first search engine result page (SERP) of Ask Jeeves, Google, MSN Search, and Yahoo were captured. 84.9% of the results were unique to one search engine, while only 1.1% were shared by all search engines. Additionally, the low overlap between search engines also manifests in the ranking since only 7% of the top results were similar. This is consistent with previous studies that reported low overlap in ranking (Bar-Ilan, 2005; Bar‐Ilan et al., 2006) Subsequent studies report an increase of overlap in search results, while ranking algorithms appear to be the leading cause for differences in the presentation of the results. Bilal and Ellis (2011) compared major web search engines (Google, Bing, Yahoo) with search engines specifically designed for children (Ask Kids, Yahoo Kids) for queries of various lengths. Bing and Yahoo shared the same overlap with Google for nearly all queries, ranging between 22% and 40%. In contrast, Yahoo Kids had mostly unique results. However, the disparate relevance ranking between Bing and Yahoo suggests different ranking algorithms. Cardoso and Magalhães (2011) measured the overlap of search result rankings based on URLs and website contents. Search results for 40,000 queries were retrieved from Google and Bing. The overlap in domains was about 29%, and Google had more exclusive domains. When looking at the result sets without considering the positions, the similarity between the result sets increased. This indicates that Google and Bing have different ranking preferences but index mostly the same sources. Similarly, Agrawal et al. (2016) based their overlap analysis on content but factored in the description of the result snippets. The top 10 results of 67 informational queries were collected from Google and Bing. The results indicate a high overlap between Google and Bing, with a slightly higher similarity among the top 5 results. Most recently, a study about information on Covid-19 was conducted by Makhortykh et al. (2020). The first 50 results for queries in different languages were collected from Baidu, Bing, DuckDuckGo, Google, Yandex, and Yahoo. DuckDuckGo and Yahoo shared nearly 50% of their results, whereas the other pairs remained under 25%. Google had an overlap of around 10% with Bing and a negligible one with DuckDuckGo, Yandex, Yahoo, and Baidu. Source diversity Various studies took a closer look at search results and evaluated the domains, the types of sources, as well as their diversity across result sets and search engines. Comparing Google, Live Search, and Yahoo, Thelwall (2008) reported that Yahoo returned the highest amount of different domains to a query and also returned around 10% more top-level domains than other search engines. Höchstötter & Lewandowski (2009) showed that while the sources in top results of various search engines differ, there is a concentration on some popular sources in top results. More recently, Lewandowski & Sünkler (2019) collected search results on queries related to insurance providers and identified the distribution of top-level domains. Only ten different domains were found in the first result across all queries. The five most popular domains in the top 5 results are price comparison websites, which make up 88.4% of all first results. The share of popular domains decreases according to the ranking but remains at 42.9% at position ten. Unkel and Haim (2019) observed Google search results prior to the German parliamentary elections in 2017. The study found that result lists for candidates and parties exhibit a high share of self-administered websites. In contrast, results about election facts, guidance, and issues are mainly from general interest news, government information, and privately run websites. Among the top ten domains across all search results are seven news websites which account for a quarter of all search results. By contrast, Wikipedia made up 5.4% of all search results. The high share of news and Wikipedia articles is confirmed by Steiner et al. (2022) who compared the first results for queries on debated topics in Germany across the search engines Ask, Bing, DuckDuckGo, Google, and Ixquick. All search engines positioned Wikipedia in the first rank for queries on climate change and the Transatlantic Trade and Investment Partnership (TTIP); Ask did so for almost every query. In the other search engines, news sites were placed in the first rank most of the time. However, for the topic Covid-19, the sources are more diverse. As Makhortykh et al. (2020) observed, only Yahoo incorporates recent information from legacy media, whereas Bing strongly relied on healthcare-related sources and Google highlighted government-related websites. Yandex was the only search engine that included alternative media in its top 20 search results. Especially for recent topics that lack an established knowledge basis, the choice of search engine may considerably impact what information a user gets to see. This paper is available on arxiv under CC 4.0 license. Authors: (1) Yagci, Nurce, HAW Hamburg, Germany & nurce.yagci@haw-hamburg.de; (2) Sünkler, Sebastian, HAW Hamburg, Germany & sebastian.suenkler@haw-hamburg.de; (3) Häußler, Helena, HAW Hamburg, Germany & helena.haeuessler@haw-hamburg.de; (4) Lewandowski, Dirk, HAW Hamburg, Germany & dirk.lewandowski@haw-hamburg.de. Authors: (1) Yagci, Nurce, HAW Hamburg, Germany & nurce.yagci@haw-hamburg.de; (2) Sünkler, Sebastian, HAW Hamburg, Germany & sebastian.suenkler@haw-hamburg.de; (3) Häußler, Helena, HAW Hamburg, Germany & helena.haeuessler@haw-hamburg.de; (4) Lewandowski, Dirk, HAW Hamburg, Germany & dirk.lewandowski@haw-hamburg.de. Table of Links Abstract and Introduction Abstract and Introduction Literature Review Literature Review Objectives and Research Questions Objectives and Research Questions Methods Methods Results Results Discussion Discussion Conclusion, Research Data, Acknowledgments, and References Conclusion, Research Data, Acknowledgments, and References LITERATURE REVIEW Search result overlap By the mid-1990s, researching the overlap of search results between different search engines sparked interest for the purpose of estimating the size of the Web (Bharat, 1998; Ding & Marchionini, 1996; Lawrence & Giles, 1999). The generally small degrees of overlap indicated diverse underlying databases, each of limited size. Therefore, metasearch engines that combined the results of several search engines were meant to provide an additional value (Chignell et al., 1999) and attracted further research (i.e., Meng et al., 2002). Consequently, Spink et al. (Spink et al., 2006) included the meta search engine Dogpile in their extensive study on the overlap of search results. For a collection of 12,570 queries, the results found on the first search engine result page (SERP) of Ask Jeeves, Google, MSN Search, and Yahoo were captured. 84.9% of the results were unique to one search engine, while only 1.1% were shared by all search engines. Additionally, the low overlap between search engines also manifests in the ranking since only 7% of the top results were similar. This is consistent with previous studies that reported low overlap in ranking (Bar-Ilan, 2005; Bar‐Ilan et al., 2006) Subsequent studies report an increase of overlap in search results, while ranking algorithms appear to be the leading cause for differences in the presentation of the results. Bilal and Ellis (2011) compared major web search engines (Google, Bing, Yahoo) with search engines specifically designed for children (Ask Kids, Yahoo Kids) for queries of various lengths. Bing and Yahoo shared the same overlap with Google for nearly all queries, ranging between 22% and 40%. In contrast, Yahoo Kids had mostly unique results. However, the disparate relevance ranking between Bing and Yahoo suggests different ranking algorithms. Cardoso and Magalhães (2011) measured the overlap of search result rankings based on URLs and website contents. Search results for 40,000 queries were retrieved from Google and Bing. The overlap in domains was about 29%, and Google had more exclusive domains. When looking at the result sets without considering the positions, the similarity between the result sets increased. This indicates that Google and Bing have different ranking preferences but index mostly the same sources. Similarly, Agrawal et al. (2016) based their overlap analysis on content but factored in the description of the result snippets. The top 10 results of 67 informational queries were collected from Google and Bing. The results indicate a high overlap between Google and Bing, with a slightly higher similarity among the top 5 results. Most recently, a study about information on Covid-19 was conducted by Makhortykh et al. (2020). The first 50 results for queries in different languages were collected from Baidu, Bing, DuckDuckGo, Google, Yandex, and Yahoo. DuckDuckGo and Yahoo shared nearly 50% of their results, whereas the other pairs remained under 25%. Google had an overlap of around 10% with Bing and a negligible one with DuckDuckGo, Yandex, Yahoo, and Baidu. Source diversity Various studies took a closer look at search results and evaluated the domains, the types of sources, as well as their diversity across result sets and search engines. Comparing Google, Live Search, and Yahoo, Thelwall (2008) reported that Yahoo returned the highest amount of different domains to a query and also returned around 10% more top-level domains than other search engines. Höchstötter & Lewandowski (2009) showed that while the sources in top results of various search engines differ, there is a concentration on some popular sources in top results. More recently, Lewandowski & Sünkler (2019) collected search results on queries related to insurance providers and identified the distribution of top-level domains. Only ten different domains were found in the first result across all queries. The five most popular domains in the top 5 results are price comparison websites, which make up 88.4% of all first results. The share of popular domains decreases according to the ranking but remains at 42.9% at position ten. Unkel and Haim (2019) observed Google search results prior to the German parliamentary elections in 2017. The study found that result lists for candidates and parties exhibit a high share of self-administered websites. In contrast, results about election facts, guidance, and issues are mainly from general interest news, government information, and privately run websites. Among the top ten domains across all search results are seven news websites which account for a quarter of all search results. By contrast, Wikipedia made up 5.4% of all search results. The high share of news and Wikipedia articles is confirmed by Steiner et al. (2022) who compared the first results for queries on debated topics in Germany across the search engines Ask, Bing, DuckDuckGo, Google, and Ixquick. All search engines positioned Wikipedia in the first rank for queries on climate change and the Transatlantic Trade and Investment Partnership (TTIP); Ask did so for almost every query. In the other search engines, news sites were placed in the first rank most of the time. However, for the topic Covid-19, the sources are more diverse. As Makhortykh et al. (2020) observed, only Yahoo incorporates recent information from legacy media, whereas Bing strongly relied on healthcare-related sources and Google highlighted government-related websites. Yandex was the only search engine that included alternative media in its top 20 search results. Especially for recent topics that lack an established knowledge basis, the choice of search engine may considerably impact what information a user gets to see. This paper is available on arxiv under CC 4.0 license. This paper is available on arxiv under CC 4.0 license. available on arxiv

Sonr

sonr.io

1-10 employees

Since 2021

COMPANY RANKING

SONR

EVERGREEN INDEX #9328

Sonr WIKI

Apply to get your Tech Company News Page live on HackerNoon.com today!

Sonr

sonr.io

1-10 employees

Since 2021

COMPANY RANKING

SONR

EVERGREEN INDEX #9328

RELATED COMPANIESRANK

STORIES ABOUT SONR

Sonr WIKI

Apply to get your Tech Company News Page live on HackerNoon.com today!