NotCommonis in the Blockchain Infrastructure Industry

NotCommon

notcommon

So You Want To Be On A Startup Board of Directors

Why You Should Forget Everything You Know About Problem Solving

The ideology serves as a powerful mechanism not only for uniting individuals with similar interests but also for exchanging experiences and ideas, embodying the principle that sharing is indeed caring. is a great example of how a community of enthusiasts can change the world. Its are highly impressive and we can claim that most of the Internet works on Linux’s horsepower. open source Linux usage statistics Bad programmers worry about the code. Good programmers worry about data structures and their relationships. Linus Torvalds Managing an open-source project carries significant weight, particularly as its popularity grows: it brings forth the spotlight on technical expertise and architectural choices. Popular projects could be a treasure trove of knowledge for anyone who wants to boost their skills. Let's delve deeply into a few notable projects. Why focus on open source? First of all, it embodies top-tier coding standards, integrating best practices championed by industry leaders. My firm belief lies in the notion that genuine programmers are fueled by passion and curiosity. Why not learn something new and integrate it into our daily practice? Secondly, hacking code could take your skills to a whole new level if you are consistent and tenacious. Last but not least, it could help you jump right into the project you like: find new buddies, brand yourself as a skilled developer, and get the job of your dreams. Help the community to solve their problems, and the reward will not keep you waiting. Take your cup of coffee, let’s get straight to work. Linux Kernel The very heart of the Linux system — its kernel. Indeed, this is not merely a singular project, but rather a constellation of diverse projects unified by a central idea: tying up different parts of hardware and providing a sustainable, secure API for the end users. The Linux codebase is huge and contains a lot of vendor-specific code, but it is well-organized and structured. I strongly recommend everyone interested to read . It might be a bit outdated but it will foster a solid understanding of how the kernel internals work. The kernel version I have on the table is 6.3. this Radix tree for memory management At the very beginning of the main.c we can spot definitions of some functions like . It is responsible for initializing the data structure that is used in memory management processes. radix_tree_init In the codebase, the radix_tree_root struct is an alias for a wrapper above the struct. It a lock (pthread mutex); xa_flags — the enum that tells how memory allocations will be performed; and the pointer to xa_head. XArray holds gfp_t struct xarray {
	spinlock_t	xa_lock;
/* private: The rest of the data structure is not to be used directly. */
	gfp_t		xa_flags;
	void __rcu *	xa_head;
}; typedef enum {
	GFP_KERNEL,
	GFP_ATOMIC,
	__GFP_HIGHMEM,
	__GFP_HIGH
} gfp_t; If all of the entries in the array are NULL, xa_head is a NULL pointer. If the only non-NULL entry in the array is at index 0, xa_head is that entry. If any other entry in the array is non-NULL, xa_head points to a xa_node — the internal struct that handles offset, counts, slots, and other fields. struct xa_node {
	unsigned char	shift;		/* Bits remaining in each slot */
	unsigned char	offset;		/* Slot offset in parent */
	unsigned char	count;		/* Total entry count */
	unsigned char	nr_values;	/* Value entry count */
	struct xa_node __rcu *parent;	/* NULL at top of tree */
	struct xarray	*array;		/* The array we belong to */
	union {
		struct list_head private_list;	/* For tree user */
		struct rcu_head	rcu_head;	/* Used when freeing node */
	};
	void __rcu	*slots[XA_CHUNK_SIZE];
	union {
		unsigned long	tags[XA_MAX_MARKS][XA_MARK_LONGS];
		unsigned long	marks[XA_MAX_MARKS][XA_MARK_LONGS];
	};
}; The radix tree data structure contains “nodes” that link with each other based on the int key (index). The node contains several slots, each of which can handle a pointer. The main idea here is to keep a tree wide, not deep — it is effective for fast addressing. When the function is called by a user (it can be a device driver), the following actions occur: radix_tree_insert __radix_create_tree creates a slot in the tree if it is necessary, based on the passed index which is used to check the size of the tree. Or raise an error if there is no available memory; insert_entities pass an item to the slot; some asserts check that offsets are applied correctly to the data structure. Wherever a user of the tree is going to look up a specific page, they need to look up it with an index. The lookup phase is quite efficient because of the broad nature of the structure. It is based on the key that is modified on the user side. Take a look at the for more insight into how it works. article Users of this data structure include GPU and network device drivers, which maintain memory page pointers in a structured format rather than juggling them. It's like having a pocket dimension! This is where Linus's earlier quote comes into play. Consider the essence of the data you handle; aim for efficiency, compactness, and cleanliness in its usage, regardless of the language or project you're involved in. In 2021 Liam Howlett (no, another Liam) and Matthew Wilcox introduced data structure. I encourage you to read more about the intentions, motivation, and realization details and . the maple tree here here Kubernetes Let's direct our steps to the point of the architectural design. One of the most famous orchestrators, invented by Google and spread around the world at rocket pace — Kubernetes. There are a lot of fairly new resources out there covering the internal details, for example here is a that you can start exploring as you delve deeper into the details. repository The whole point of this article is to get some useful insights that answer not just “how” but “why”. Understanding the motivation is crucial, in my opinion, that is how we can “catch a mindset” to re-implement all those cool staff in our daily job. Let’s briefly recall what the Kubernetes is. This is the system that manages containers within the applications and provides orchestration features such as scaling and high availability. Kubernetes’ main components are the API service, the scheduler, etcd — the key-value storage, the controller service, and the agent service — “kubelete” on each node to manage containers. Guys from developed it to utilize the compute resources of their cloud . Their main intention was a “great container management system”. Google as efficiently as possible Architectural highlights The very first thing I’d like to highlight is the distributed nature of execution. To synchronize all of the user’s entities states the orchestrator uses etcd — the key-value storage which information in a tree data structure. It is a common pattern that presents a to manage different states between multiple entities, with one slight difference: the storage is also used to “instruct” all of the components about the final cluster’s state. keeps So the whole system uses the storage as a source of truth to keep maintaining the desired state of a cluster. source of truth One more nice architectural decision is the — container runtime interface. CRI It allows Kubernetes to work not just with Docker but many others (CRI-O, Dockerd) by defining the , just like one from RFC. The kubelet agent utilizes GRPC to connect with a runtime, so in case one would like to implement its own runtime, they should implement the exact interface on their side. The concept aims to loosen the connection between the kubelet agent and a runtime which manages containers. interaction protocol At the end let’s briefly look at the “sidecar” pattern that was presented by third-party services like and has recently been within the core Kubernetes functionality. Istio implemented The whole idea is to embed a service to the main application that helps to handle some specific cases such as proxy or load-balancing. Kubernetes manages this container as a whole different entity of a pod: the container can the entire pod status; restarting and updating separately of the main application; resources with the main app. influence share The main point here is that the main application may know nothing about the sidecar, while the behavior of the entire service can be changed. Bless Open Source, we how new functionality was implemented by the community. can trace PostgreSQL PostgreSQL is not just a database but a whole platform. After the client connects to the server a new is established. The query goes through many while executed: parsing, the rewriting system, planning and executing. backend process stages The PSQL server has a bunch of that allow developers to interact with different internal components. For example, we can our scan provider aimed, for some optimizations at each stage. Or create an . Even though the overall process required for specific knowledge and the documentation could be a bit knotty, interfaces write index add-on the loosely coupled architecture empowers developers to vastly expand functionality. There are many that we can use within the platform and the “bloom index” is one of them. extensions realization Bloom index I faced a problem of growing indexes back in the day. The task was to implement the custom search filter: every column in the database and the combination of it could be a key for a search. Building indexes for each and every column could be a bad decision because of the complexity and size. For example, if we are sure that item==2 is False there is no reason to check all other fields. I remembered the — the probabilistic data structure that could return a false-positive result (because of hash collision) and be quite sure when the data is absent. As I’ve mentioned earlier PSQL has an extension for this data structure. The very first idea that comes to mind is to reduce the dimension of possible variations removing the query values that we are sure are not in the database. Bloom filter Bloom filter in a nutshell — it’s just a bit array. Every item would be hashed with a certain amount of different functions that translates it to some index. That index will set the exact bit in the filter’s array. In the case of 3 different functions, an item would set 3 different bits. At the lookup phase, the query would be also hashed and compared within the bits in the array. If at least one of the bits is not set, we can be sure the item is absent. If all of them were set, it is still possible that the item was not placed — that is because each bit we have found could refer to other items — the collision takes place. Check out this great of how it works within the examples. explanation To start using the bloom index within the PSQL we need to create it properly: CREATE INDEX bloom_idx_bar ON foo.bar USING bloom (id,dept_id,zipcode) WITH (length=80, col1=4, col2=2, col3=4); where “length” — is the length of the array of bits for each row; and the “col” numbers — describe how many bits would be occupied by the column (how many functions would be applied). By increasing these numbers we can reduce false-positive shots. Of index creation in PSQL is a bit complicated, we should understand internal data structures and protocols; let’s try to make the observation short and sweet. The function promotes the data structure’s functions as an IndexAmRoutine pointer — a pointer to the , that represents an API for the index access methods. blhandler struct The function creates the new index: allocate memory pages, scan the table for tuples that will be indexed and in the structure. blbuild BloomState typedef struct BloomState
{
	FmgrInfo	hashFn[INDEX_MAX_KEYS];
	Oid			collations[INDEX_MAX_KEYS];
	BloomOptions opts;			/* copy of options on index's metapage */
	int32		nColumns;

	/*
	 * sizeOfBloomTuple is index-specific, and it depends on reloptions, so
	 * precompute it
	 */
	Size		sizeOfBloomTuple;
} BloomState; When we insert data to the index, we pass it in the form of an array within the array of booleans that marks NULL values to skip it at the indexing phase. The BloomFormTuple function forms the BloomTuple struct from the data — this is the part where the data would be signed. The tuple would be passed to a memory page after that phase. BloomTuple *
BloomFormTuple(BloomState *state, ItemPointer iptr, Datum *values, bool *isnull)
{
	int			i;
	BloomTuple *res = (BloomTuple *) palloc0(state->sizeOfBloomTuple);

	res->heapPtr = *iptr;

	/* Blooming each column */
	for (i = 0; i < state->nColumns; i++)
	{
		/* skip nulls */
		if (isnull[i])
			continue;

		signValue(state, res->sign, values[i], i);
	}

	return res;
} At the scan phase the forms the struct which contains the index’s data and the makes the TIDBitmap struct from it — the internal data structure that holds item pointers — reading BloomState page by page. blbeginscan IndexScanDesc blgetbitmap I hope the explanation isn't too confusing, PSQL internals are quite specific. Bloom filter is a pretty common data structure, PSQL uses the same approach for some purposes. It is also broadly used in many different situations: from cache-hitting cases to implementations. internal kernel packet filter More to know In the article, I’ve just lifted the veil of the shiny open-source world. There are many more great implementations you could meet in the wild: and — if you’d like to catch the magic of the “fair” resource allocation; — to understand the simultaneous editing in Google Docs. CFS CFQ operational transform and CRDT The whole software world is in motion: sidecars and operators to be the new model for software delivery; eBPF is in a to change the service-mesh post. claimed hurry Take a look at the elegance of the great things’ internals.

Algorithms in the Wild: Reusable Insights and Strategies from the Open Source Community

is a seasoned entrepreneur and fintech specialist and is here to shed light on the technicalities surrounding international bank transfers and their impact on businesses, especially SMEs. Iurii Gugnin With over a decade and a half of experience in the field, Gugnin is currently spearheading the growth of Evita, an American that is revolutionizing the way companies handle international transfers. startup In a short span of six months, Evita has achieved a remarkable turnover of $15 million per month, proving its profitability and effectiveness. The startup is instrumental in facilitating overseas purchases of real estate, automobiles, and equipment for foreign clients while also aiding American small and medium-sized businesses in importing goods from abroad at a fraction of the cost. Unlike traditional banking institutions such as Bank of America and Chase, which levy a 3-5% charge on transfers, Evita operates on a minimal commission of 0.4-1.5%. This approach not only fuels the growth of the US economy but also aids in re-establishing the country's dominance in . global exports Join us as Iurii Gugnin delves into the challenges faced by the U.S. in expanding its exports, the reasons behind the high costs of international wire transfers, and potential solutions to these issues. Q: With a wealth of experience in fintech since 2009, could you share some insights about your journey with international payments? A: My journey with international payments has been quite diverse, encompassing roles as an entrepreneur, consultant, and individual user. I've had the privilege of traveling to over 100 countries, living in several for extended periods, and consistently encountered a common issue: the incompatibility of bank cards when moving to a new location. For instance, a Thai card was unusable in the States, and a Dubai card was ineffective in the UK. Despite Visa and MasterCard's global presence, their usability is conditional. In my entrepreneurial journey, I've faced similar challenges. I established an international business, secured licenses in Europe and the CIS countries, and am currently in the process of obtaining licenses in America. One of my notable ventures is the Karma investment platform, which has successfully built a community of over 8,000 investors, attracted more than 500 business borrowers, and structured a $250m deal with a major bank. However, the process of making international transfers with currency exchange was often marred by high fees and lengthy waiting periods. In addition to this, I've advised five countries on digital asset legalization, engaging with central banks to discuss payment structuring and the integration of digital money to enhance transfer transparency. Drawing from these experiences, I founded Evita, a US-based company, in 2021. After a year of product-market fit exploration and several pivots, we found our niche in making international money transfers faster and cheaper, leading to rapid growth. Q: The Federal Reserve reports that the average cost of money transfers from the States to other countries is now about 5.6%, even higher than in 2022. Why are these fees so high? A: The high fees can be attributed to what I call the 'imperial syndrome' prevalent in the States. There's a common belief that dollars are universally accepted, negating the need for other currencies. Mainstream banks like Wells Fargo and Chase primarily operate in dollars in the United States, and the SWIFT system also predominantly uses USD. Growing up in Asia, I was accustomed to carrying different currencies and exchanging them with ease, a practice not common in the US. Despite my knowledge and net worth, it took me three months to open a EUR account in the US, and even then, there were significant usage restrictions. This is a challenge that many local entrepreneurs face. There is a niche segment of VIP banks, such as HSBC, Morgan Stanley, and JP Morgan, where corporations or wealthy individuals can open accounts in different currencies. These banks offer competitive exchange rates and reasonable transfer fees, but they are not designed for the mass market. In essence, there's an oligopoly of 3-4 leaders in the US banking sector who set their own prices. They've decided to charge 3-5% for international transfers, and most customers, unaware of how exchange rates are determined or where to find current information, have no choice but to comply. Q: In many Asian countries, the transfer fee is significantly lower than in the United States. For instance, in South Korea, it's only 3.55%. Can you shed some light on why this is the case? A: Asia, with its multitude of island nations, has a unique culture of multi-currency transactions. However, even a seemingly small fee of 3.5% can add up quickly. Consider purchasing a Mercedes-Benz G-Class abroad for 300,000 EUR. The commission alone would amount to a staggering 10,500 EUR. There are companies like Revolut or Wise that cater to smaller transactions, such as a $1,000 payment for freelance services. In these cases, an additional percentage doesn't seem as daunting. But in the B2B sector, the stakes are much higher. Transactions often involve tens or even hundreds of thousands for equipment, material supplies, and furniture. On our platform, we've even facilitated the purchase of a thoroughbred horse from Germany with an impressive pedigree. The price tag was 120,000 EUR! In such scenarios, even half a percent can make a significant difference. Q: The process of cross-border transfers often results in high costs and extended durations. Would you provide some insight into the underlying reasons for this? A: Certainly, let's consider a hypothetical scenario. Imagine two individuals who wish to open dollar accounts in a bank in Mongolia and another in Korea. The question arises, how can this be achieved? The challenge lies in the fact that to handle non-cash dollars, both banks require a license from the United States. Acquiring such a license is a daunting task for foreign financial institutions. As a workaround, these Mongolian and Korean banks would typically open correspondent accounts in U.S. banks, say Wells Fargo and Citi, respectively. Within these accounts, virtual folders are created for specific users. Now, let's assume a client from Mongolia wishes to send dollars to a friend in Korea via SWIFT. As I mentioned earlier, the currency is held in American banks. The initiating bank, Wells Fargo, would first verify the sender and recipient of the transfer. Following this, it would execute the transfer within the U.S., to Citi. The receiving bank, Citi, would then confirm that the Korean bank has an account with them and that the intended recipient is indeed a client of theirs. This process of verification and confirmation takes time, and during this period, transaction fees accumulate. Hence, the perceived delay and cost associated with cross-border transfers. Q: So, if I understand correctly, Mongolian and Korean banks might not directly interact with US dollars, but rather through an intermediary. Does this additional layer extend the transfer time? A: Yes, this is a common practice in many countries. A local bank might open an account with a Korean bank, which in turn has an account with a US bank. Consequently, the money passes through several "hands" of correspondent banks, each taking a slice of the transaction as commission. Moreover, when funds are sent through this complex chain, the transaction details can be extensive, often filling an entire page of text. This complexity significantly increases the likelihood of errors, which further extends the transfer time. But that's not all. Customers have come to expect SWIFT transfers to be a lengthy process. For instance, when money is sent from the US to Latin America, the CIS, or Asia, local banks might reflect the transfer in the customer's account a day or even a week after the funds have arrived. During this period, the banks essentially use the customer's money for free, earning interest on the deposits. Q: Would it be accurate to say the complexities associated with international transfers contribute significantly to the U.S. losing its leadership in global exports? A: I believe it's a crucial factor. The U.S.'s reliance on the dollar has had unintended consequences. In 2012, the U.S. was surpassed by China in terms of global exports, and by 2022, the Chinese yuan had become in transaction volume. one of the top five currencies While the dollar still leads, the use of the yuan from 2019 to 2022. nearly doubled Despite the slowdown in China's economy due to low birth rates, it continues to progress, driven by momentum. Furthermore, China frequently devalues its currency, which benefits the country as expenses are nullified while income remains in dollars. Consequently, Chinese goods remain competitive in the global market. In contrast, the U.S. faces high costs in various sectors, including real estate, transportation, internet, and food. These costs inflate the price of goods, making them less attractive overseas. Unique products like Boeing airplanes or Nvidia chips are exceptions. Additionally, businesses bear the burden of a 5%+ commission for payment transfers and interest rates on loans. Q: What solution does Evita offer? A: Fast and cheap international B2B transfers. We have a large community of exporters and importers who need to regularly transfer large sums to other countries. We also have our accounts in the States, in the UK, in the EU, and soon to be in China. This allows us to offset counter-flows of money. Q: Could you provide some insight into how the counterflow system operates? A: Imagine a scenario where one company needs to transfer $1 million from the US to Europe today, and another company needs to send the same amount from Europe to the US. In this abstract situation, we at Evita can accept the dollars into our American account and simultaneously transfer euros from our local European account and vice versa. This makes all transactions swift as they are conducted within the US and EU respectively. Moreover, we save on currency conversion because we don't convert the currencies. Our counterflow system is supported by a community of importers, exporters, logistics operators, and entrepreneurs from various countries. My extensive experience in fintech has been instrumental in bringing this community together. Large banks also have systems that allow for such internal clearing, which significantly increases the profitability of transactions. However, clients are often not informed about this and are charged hefty commissions for transfers. In reality, banks do not change the currency but simply adjust the figures on the accounts. At Evita, we choose not to keep all the profit for ourselves. Instead, we share a portion of it with our clients in the form of a discount. As a result, the user will pay a commission of only 0.4-1.5%, depending on the transaction volume, instead of the usual 5% or even 3%. Essentially, we democratize access to clearing, which is currently monopolized by banks, and save up to 90% of money on currency exchange. According to the , U.S. small businesses export goods and services worth $341 billion annually. We estimate that products like Evita could save Americans between $7-16 billion a year. SBA Q: What happens if the counterflows don't align? A: In such cases, we resort to transferring money through digital wallets using stablecoins. We maintain legal and institutional accounts on regulated exchanges like Kraken. Cryptocurrency transfers offer a faster and more cost-effective alternative to traditional bank wires, taking just a minute instead of two days and costing 50 cents instead of $45. Q: Your primary focus is on small businesses, correct? A: Our focus is on small and medium-sized businesses. However, it's important to note that these businesses often engage in quite large transactions. While we don't cater to corporations on the scale of Apple, our clients are primarily involved in buying and selling cars. Currently, we're exploring the luxury furniture niche. Our services are also frequently used for payments related to equipment and charter flights, which can cost up to $200,000. Q: Is it accurate to say Evita also assists businesses in addressing their working capital issues? A: That's correct. It's a common scenario where a business has paid for a product and is simply waiting for its delivery. Many suppliers only ship products after receiving payment. If the transfer is made via SWIFT, the customer may have to wait an additional two weeks. During this period, the money is essentially "blocked" in the system. However, if the funds are transferred within a day, businesses can turn over their working capital more frequently, leading to increased earnings at the end of the year. This is often more significant than the profitability of a single transaction. We aim to shorten the cycle and increase the number of "turnovers.” Moreover, businesses can avoid taking out loans and paying high-interest rates. Currently, the interest rate on a small business loan . Not all businesses can secure these loans from banks due to stringent requirements, leading to the emergence of loan sharks who offer loans at annual rates of 100-200%. ranges from 6% to 45% By freeing up working capital, Evita could potentially save US exporters around $20 billion a year at the very least. Given that in the U.S., it's crucial to help them save money. small businesses are the backbone of job creation Q: Rounding off the interview, what do you think the future holds for the international payments market? A: The future of international payments, in my view, may present more challenges. Despite the technological capability for instant and affordable global transfers existing for over two decades, two major obstacles persist. Secondly, there is a global trend of increasing governmental control over business operations. European banks, for instance, annually on Anti-Money Laundering (AML) and related activities. These costs are inevitably passed on to consumers through service fees. spend an average of 14.25 million euros Collectively, Europeans per year on AML measures. However, these efforts only uncover less than 1% of actual criminal activity, while 99.99% of individuals and businesses bear the brunt of compliance requirements. spend 140 billion euros As regulatory measures continue to intensify, domestic and ecosystem-specific money transfers will become the norm. Consequently, services like ours that facilitate these transactions are poised to gain popularity.

Interview With Iurii Gugnin: Boosting U.S. Export Competitiveness With Cheaper Remittances

Emphatic words must stand in emphatic positions

Scientific American Supplement, No. 717, September 28, 1889, by Various, is part of the HackerNoon Books Series. You can jump to any chapter in this book . ALKALI MANUFACTORIES. here ALKALI MANUFACTORIES. When the alkali, etc., Works Regulation Act was passed in 1881, it was supposed that the result would be that the atmosphere in the districts where such works are situated would be considerably improved, and, consequently, that vegetation would have a better chance in the struggle for existence, and the sanitary conditions of human dwellings would be advanced. In all these respects the act has been a success. But perhaps the most notable result is the effect which the act and those which have preceded it have had upon the manufactures which they control. This was not anticipated by manufacturers, but now one of the principal of them (Mr. A. M. Chance) has stated that "Government inspection has not only led to material improvement in the general management of chemical works, but it has also been in reality a distinct benefit to, rather than a tax upon, the owners of such works." This expression of opinion is substantiated by the chief inspector under the act, whose report for last year has recently been laid before the local government board. There are 1,057 works in the United Kingdom which are visited by the inspectors, and in only two of these during 1888 did the neglect to carry out the inspectors' warnings become so flagrant as to call for legal interference; viz., in the case of Thomas Farmer & Co. (limited), Victoria Docks, E., who were fined 20 and costs for failing to use the "best practicable means" for preventing the escape of acid gas from manure plant; and in the case of Joseph Fison & Co., Bramford, who were fined 50 and costs for excessive escape of acid gas from sulphuric acid plant. There were seven other cases, but these were simply for failure to register under the act. l. l. It is very evident, therefore, that from a public point of view the act is splendidly successful, and from the practical or scientific side it is no less satisfactory. Of the total number of chemical works (1,057) 866 are registered in England, 131 in Scotland, and 44 in Ireland—a decrease in the case of Scotland of 8, and in Ireland of 2 from the previous year, while England has increased by 1. This must not, however, be taken as a sign of diminished production, because there is a tendency for the larger works to increase in size and for the smaller ones to close their operations. The principal nuisances which the inspectors have to prevent are the escape of hydrochloric acid gas from alkali works and of sulphurous gas from vitriol and manure works. The alkali act forbids the manufacturer to allow the escape of more than 5 per cent. of the hydrochloric acid which he produces, or that that acid must not exist to a greater extent than 0.2 grain in 1 cubic foot of air, steam, or chimney gas which accompanies. The inspectors' figures for last year show that the percentage of the acid which escaped amounted to only 1.96 of the total produced, which is equal to 0.089 grain per cubic foot, and much below the figures for previous years. The figures in regard to sulphurous gas are equally satisfactory. The act allows 4 grains of sulphuric anhydride (SO3) per cubic foot to escape into the air, and last year's average was only 0.737 grain, or less than a fifth of the limit. Of course it is now the aim of the Leblanc alkali manufacturers to reduce the escape of hydrochloric acid to the lowest possible amount, as their profits depend solely upon the sale of chlorine products, soda products being sold at a loss. In this connection it is interesting to note that the amount of common salt manufactured in the United Kingdom in 1888 was 2,039,867 tons, and of this nearly 600,000 tons were taken by Leblanc soda makers, and over 200,000 tons by the ammonia-soda makers. The figures are very largely in excess of previous years, and indicate a gratifying growth in trade. The salt used in the Leblanc process yields the hydrochloric acid, and that in the ammonia-soda method none, so that we may put down the theoretical production of acid as 380,000 tons, 7,600 tons of which was allowed to escape. What was a mere trace in the chimney gases amounts, therefore, to a good round figure at the end of a year, and if it were converted into bleaching powder it would be worth nearly 150,000 These figures are, it should be understood, based on theory, but they serve to show to what importance a gas has now reached which twenty-five years ago was a perfect incubus to the manufacturers, and wrought desolation in the country sides miles and miles around the producing works. There has long been an expectation that the ammonia-soda makers would add the manufacture of bleaching powder to their process, but they appear to be as far as ever from that result, and meanwhile the Leblanc makers are honestly striving to utilize every atom of the valuable material which they handle. Hence the eagerness to recover the sulphur from tank waste by one or other of the few workable processes which have been proposed. l. This waste contains from 11 to 15 per cent. of sulphur, and when it is stated that the total amount of tank waste produced yearly is about 750,000 tons, containing about 100,000 tons of sulphur, it will be seen how large is the reward held out to the successful manipulator. Moreover, the value of the sulphur that might possibly be saved is not the only prize held out to those who can successfully deal with the waste, for this material is not only thrown away as useless, but much expense is incurred in the throwing. In Lancashire and in other inland districts land must be found on which to deposit it, and the act of depositing is costly, for unless it is beaten together so as to exclude the air, an intolerable nuisance arises from it. The cost of haulage and deposit on land varies, according to the district, from 1s. to 1s. 6d. a ton. In Widnes it is about 1s. In the Newcastle district the practice is to carry this material out to sea at a cost of about 4d. a ton. Mr. Chance's process for the recovery of sulphur from the waste signalizes the centenary of the Leblanc process; Parnell and Simpson are following in his wake, and lately Mr. F. Gossage, of Widnes, has been working on a process for the production of alkali, which enables him to save the sulphur of the sulphuric acid. In his process a mixture of 70 parts Leblanc salt cake (sulphate of soda) and 30 parts common salt is mixed with coal and heated in a furnace, and so reduced to sulphide of sodium. The resulting "ash" is then dissolved in water and exposed to the action of carbonic acid, when sulphureted hydrogen is given off, to be dealt with as in Mr. Chance's sulphur process, while bicarbonate of soda is formed and separates by precipitation from the solution of undecomposed common salt. Ere long it is expected this new method will be in active operation in some Leblanc works, the plant of which will, in all probability, be utilized. It has these great advantages: The absence of lime, the recovery of the sulphur used in the first instance and the consequent absence of the objectionable tank waste. Thus a bright promise is held out that the days of alkali waste are numbered, and that the air in certain parts of Lancashire will be more balmy than it has been in the memory of the oldest inhabitant.— Chemist and Druggist. About HackerNoon Book Series: We bring you the most important technical, scientific, and insightful public domain books. This book is part of the public domain. Various (2006). Scientific American Supplement, No. 717, September 28, 1889. Urbana, Illinois: Project Gutenberg. Retrieved https://www.gutenberg.org/cache/epub/17755/pg17755-images.html This eBook is for the use of anyone anywhere at no cost and with almost no restrictions whatsoever. You may copy it, give it away or re-use it under the terms of the Project Gutenberg License included with this eBook or online at , located at . www.gutenberg.org https://www.gutenberg.org/policy/license.html

ALKALI MANUFACTORIES

Generative AI is all the rage. In this modern age, the numerous applications of Generative AI in different sectors, including Business Intelligence and Data Analytics are undeniable. Companies are incorporating AI in their workflow to automate tasks, produce content, scale their profit, and do more in less time. This is a significant increase from 1% of data it accounted for a few years ago, says the same report. Generative AI is getting so widespread that by 2025, it will account for , reports Gartner. 10% of all produced data For product-based businesses, Gen AI can create personalized experiences for customers by checking historical data. Finance companies can analyze market data to propose new trading strategies. Every industry, if it involves data analysis, will benefit from the new and emerging Artificial Intelligence technologies. In this article, we’ll explore 5 main use cases of generative AI in business intelligence and data analytics and how real companies are making use of it. Let’s dive in. 1. Automate Analytics A lot of tasks in business intelligence and data analytics can be repetitive and cumbersome. While you could get them done using automated scripts, again, that requires a high level of coding knowledge. Moreover, you need to write scripts every time you need to . If you decide to change your approach when doing the analysis, you’ll need to change the script. automate an analytical task Data collection, for example, is one of the most time-consuming tasks. You need to find the right data sources, get all your Excel files together, search for the relevant information, and start your analysis. A more innovative use case of generative AI in data analytics is to automate the data extraction and analysis process. With most modern AI chatbots, you can just ask them to create automation scripts for you, fully personalized and tailored to your specific needs. When collecting data, you can ask the AI tool to filter out relevant data by inputting some parameters. You can set up an automatic notification system so that if there is any change in customer behavior or sales, you can use that data immediately and take action. This can save you time, and resources, and in turn, save you from potential revenue loss. Seek.ai is an example of a generative AI tool that can automate data extraction and analytics through AI-powered database queries. It enables you to query your database, , through trained deep-learning models that convert natural human language into precise and executable SQL code, that searches through your business data and retrieves insights. including Microsoft SQL servers This way, you can free up more time for your data science teams, while increasing the ROI of your business data. 2. Data Preparation Data preparation involves many complex steps such as data collection, discovery & profiling, cleansing, structuring, transformation, and validation. This makes the process of preparing data often complicated. When collecting data from different sources, there can be inconsistencies in the quality and accuracy of the data. Not to mention, much of this data can be unusable and irrelevant, so you have to remove them. This can take away a lot of valuable time and resources and have a bad effect on workforce productivity. In fact, data scientists spend labeling, cleaning, and organizing data, reports Crowd Flower, a user data enrichment platform. 51% of their time Imagine how much time and energy your business could save if these tasks could be solved by AI. Generative AI can take care of many such data preparation tasks as tagging, segmentation, classification, enrichment, and anonymization. Some of these models have augmented data preparation abilities. That means you can automate profile data, weed out errors and irrelevant data, and perform data cleansing, transformation, and enrichment. Many Business Intelligence vendors offer generative AI technologies with data preparation automation features. These can streamline how you as a self-service BI user can do advanced data analytics work, even without expert knowledge. The team at Snorkel leveraged Vertex AI to extract important data from complex documents. Snorkel Flow uses to programmatically label the data. These can be PDFs, unstructured text, HTML data, richly formatted documents, or conversational text. data-centric AI workflow They use Vertex AI for monitoring the model performance. In case of a data drift, they can quickly adjust parameters and regenerate training models. 3. Predictive Analysis Predictive analysis involves analyzing your current data to predict the future outcome. Data Analytics can be divided into four types, and predictive analysis deals with the future. It uses , machine learning, decision trees, cluster models, etc., for analytics purposes. statistical models For many businesses, this can be a complicated process. and coming to a future-proof conclusion based on that can be a daunting process. Keeping updated with the trends, collecting relevant data in vast amounts, feeding them to the analysis models, Even if you go through the process, that doesn’t necessarily guarantee an accurate result. On the contrary, there’s more chance of being wrong which results in big losses for companies. Generative AIs are trained on a huge amount of datasets. You can also personalize the data that you train it on, making these tools more effective in analyzing the current situation and making better future decisions. These tools can identify patterns and trends more easily than humans and traditional Data Analytics tools. You can also train and improve the accuracy of your existing predictive models using Generative AI. One way to do that is by generating synthetic data and testing your models more realistically and robustly. With such tools in hand, you can better optimize your business operations and increase efficiency. Google’s AI tool, for example, can by scanning the retina. predict cardiovascular risks The algorithm was presented with patients who suffered from the disease and those who didn’t. It made the correct prediction 70% of the time. This is slightly less than the usual testing method which involves blood tests. 4. Risk Management Evaluating risks is essential for any business to become successful. Not using the correct risk metrics, wrongfully assessing risks, selecting unreliable data sources, or incorrectly communicating the identified risks can make your company face pitfalls. Moreover, companies need to monitor risks 24/7. Traditional tools like spreadsheets are not only limited but also require constant adjustments. This takes up employee time and resources and hurts productivity. Another common problem is risk treatment. You may have identified and assessed any potential risks, but how would you manage them? While regular BI tools can help, you need to be aware of their usage in risk management. So risk management is another one of the best use cases of generative AI in business intelligence. You need to make informed decisions at the correct time to avoid risks. Generative AI can help you deal with risk management situations. AI technologies in business intelligence can locate potential business risks and generate valuable reports outlining them. For example, a financial company can use generative AI to detect any kind of fraud and financial crimes. One of the main use cases of generative AI for risk management is simulation. You can create an imaginary scenario, analyze the risks involved, and build a mitigation strategy in advance. This can also be useful for testing your current strategy and tweaking it to handle any edge cases that you probably didn’t consider. Stripe to offer a better user experience and detect fraud. took advantage of GPT-4 After testing 50 potential uses of this model in their business, they found 15 strong use cases. They improved their support customization, started answering questions about support, summarized documents, and detected any fraudulent activities. 5. Generating Visual Data Data visualization is a prominent use case of generative AI in data analytics. Business intelligence and data analytics tools can be limited when turning data into a visual representation. Some tools can only . They may not output the visual the way you want it to be. The result can’t be further personalized or customized. plot traditional charts and graphs For inexperienced eyes, reading and comprehending can prove to be hard even if it’s visual data. It can also be difficult to use advanced tools for someone who isn’t rather tech-savvy. So even with powerful tools, the end goal isn’t served. Generative AI changes how you generate visualized data, and images, and model your insights. Additionally, unlike traditional tools, you can personalize the output you receive using advanced prompts, making the visualization tailored to your needs. This benefits you as well as your customers. That’s because of the user-friendly nature of these images. On top of that, you can make the visualizations interactive to assess data in real-time. It’s easier to generate visualization and get hidden insights from them when you can input natural language prompts for them. Even if you don’t have coding experience or can’t understand data visualization, generative AI can make you comprehend them easily. Telus, a publicly traded holding company, to improve customer experience and performance analytics. used HEAVY.ai They used interactive visualization features of the tool to find important insights from customer data which enables them to create upsell opportunities. As a result, they were able to build wireless services in profitable locations just by querying, filtering, and visualizing the geo-temporal data. Conclusion Generative AI can completely change your Business Intelligence landscape. From data collection, data analysis, augmenting datasets, and addressing data scarcity, to improving model accuracy, getting reliable insights, generating visualizations, and making business decisions, just to name a few. These AI tools will help you build a business with a personalized experience for both you and your consumers. Interestingly, McKinsey reports that Generative AI has the potential of adding across the 63 business use cases. $2.6 trillion to $4.4 trillion every year By now, you should have a clear idea of the benefits and use cases of Generative AI in business intelligence and data analytics. Adopting AI to your business will not only streamline your whole workflow but also remove any human limitations you had previously. With more accurate data in your hands, you can unlock a world of new opportunities. So now is the time to stop pushing back, and start using these technologies to save more business resources and earn more revenue.

5 Main Uses of Generative AI in Business Intelligence & Data Analytics

On War by Carl Von Clausewitz, is part of the HackerNoon Books Series. You can jump to any chapter in this book here . Inundations Book VI: Chapter XX B. Inundations Now we have still to consider inundations. As defensive means and also as phenomena in the natural world they have unquestionably the nearest resemblance to morasses. They are not common certainly; perhaps Holland is the only country in Europe where they constitute a phenomenon which makes them worth notice in connection with our object; but just that country, on account of the remarkable campaigns of 1672 and 1787, as well as on account of its important relation in itself to both France and Germany, obliges us to devote some consideration to this matter. The character of these Dutch inundations differs from ordinary swampy and impassable wet low lands in the following respects: 1. The soil itself is dry and consists either of dry meadows or of cultivated fields. 2. For purposes of irrigation or of drainage, a number of small ditches of greater or loss depth and breadth intersect the country in such a way that they may be seen running in lines in parallel directions. 3. Larger canals, inclosed by dykes and intended for irrigation, drainage, and transit of vessels, run through the country in all possible directions and are of such a size that they can only be passed on bridges. 4. The level of the ground throughout the whole district subject to inundation, lies perceptibly under the level of the sea, therefore, of course, under that of the canals. 5. The consequence of this is, that by means of cutting the dams, closing and opening the sluices, the whole country can be laid under water, so that there are no dry roads except on the tops of the dykes, all others being either entirely under water or, at least, so soaked that they become no longer fit for use. Now, if even the inundation is only three or four feet deep, so that, perhaps, for short distances it might be waded through, still even that is made impossible on account of the smaller ditches mentioned under No. 2, which are not visible. It is only where these ditches have a corresponding direction, so that we can move between two of them without crossing either, that the inundation does not constitute in effect an absolute bar to all communication. It is easy to conceive that this exception to the general obstruction can only be for short distances, and, therefore, can only be used for tactical purposes of an entirely special character. From all this we deduce 1. That the assailant’s means of moving are limited to a more or less small number of practicable lines, which run along very narrow dykes, and usually have a wet ditch on the right and left, consequently form very long defiles. 2. That every defensive preparation upon such a dam may be easily strengthened to such a degree as to become impregnable. 3. But that, because the defensive is so hemmed in, he must confine himself to the most passive resistance as respects each isolated point, and consequently must look for his safety entirely from passive resistance. 4. That in such a country it is not a system of a single defensive line, closing the country like a simple barrier, but that as in every direction the same obstacle to movement exists, and the same security for flanks may be found, new posts may incessantly be formed, and in this manner any portion of the first defensive line, if lost, may be replaced by a new piece. We may say that the number of combinations here, like those on a chessboard, are infinite. 5. But while this general condition of a country is only conceivable along with the supposition of a high degree of cultivation and a dense population, it follows of itself that the number of passages, and therefore the number of posts required or their defence, must be very great in comparison to other strategetic dispositions; from which again we have, as a consequence, that such a defensive line must not be long. The principal line of defence in Holland is from Naarden on the Zuyder Zee (the greater part of the way behind the Vecht), to Gorcum on the Waal, that is properly to the Biesbosch, its extent being about eight miles. For the defence of this line a force of 25,000 to 30,000 was employed in 1672, and again in 1787. If we could reckon with certainty upon an invincible resistance, the results would certainly be very great, at least for the provinces of Holland lying behind that line. In 1672 the line actually withstood very superior forces led by great generals, first Condé, and afterwards Luxembourg, who had under their command 40,000 to 50,000 men, and yet would not assault, preferring to wait for the winter, which did not prove severe enough. On the other hand, the resistance which was made on this first line in 1787 amounted to nothing, and even that which was made by a second line much shorter, between the Zuyder Zee and the lake of Haarlem, although somewhat more effective, was overcome by the Duke of Brunswick in one day, through a very skilful tactical disposition well adapted to the locality, and this although the Prussian force actually engaged in the attack was little, if at all, superior in numbers to the troops guarding the lines. The different result in the two cases is to be attributed to the difference in the supreme command. In the year 1672 the Dutch were surprised by Louis XIV., while everything was on a peace establishment, in which, as is well known, there breathed very little military spirit as far as concerned land forces. For that reason the greater number of the fortresses were deficient in all articles of material and equipment, garrisoned only by weak bodies of hired troops, and defended by governors who were either native-born incapables, or treacherous foreigners. Thus all the Brandenburg fortresses on the Rhine, garrisoned by Dutch, as well as all their own places situated to the east of the line of defence above described, except Groningen, very soon fell into the hands of the French, and for the most part without any real defence. And in the conquest of this great number of places consisted the chief exertions of the French army, 150,000 strong, at that time. But when, after the murder of the brothers De Witt, in August 1672, the Prince of Orange came to the head of affairs, bringing unity to the measures for national defence, there was still time to close the defensive line above-mentioned, and all the measures then adopted harmonised so well with each other that neither Condé nor Luxembourg, who commanded the French armies left in Holland after the departure of the two armies under Turenne and Louis in person, would venture to attempt anything against the separate posts. In the year 1787 all was different. It was not the Republic of seven united provinces, but only the province of Holland which had to resist the invasion. The conquest of all the fortresses, which had been the principal object in 1672, was therefore not the question; the defence was confined at once to the line we have described. But the assailant this time, instead of 150,000 men, had only 25,000, and was no mighty sovereign of a great country adjoining Holland, but the subordinate general of a distant prince, himself by no means independent in many respects. The people in Holland, like those everywhere else at that time, were divided into two parties, but the republican spirit in Holland was decidedly predominant, and had at the same time attained even to a kind of enthusiastic excitement. Under these circumstances the resistance in the year 1787 ought to have ensured at least as great results as that of 1672. But there was one important difference, which is, that in the year 1787 unity of command was entirely wanting. What in 1672 had been left to the wise, skilful, and energetic guidance of the Prince of Orange, was entrusted to a so called Defence Commission in 1787, which although it included in its number men of energy, was not in a position to infuse into its work the requisite unity of measures, and to inspire others with that confidence which was wanted to prevent the whole instrument from proving imperfect and inefficient in use. We have dwelt for a moment on this example, in order to give more distinctness to the conception of this defensive measure, and at the same time to show the difference in the effects produced, according as more or less unity and sequence prevail in the direction of the whole. Although the organisation and method of defence of such a defensive line are tactical subjects, still, in connection with the latter, which is the nearest allied to strategy, we cannot omit to make an observation to which the campaign of 1787 gives occasion. We think, namely, that however passive the defence must naturally be at each point in a line of this kind, still an offensive action from some one point of the line is not impossible, and may not be unproductive of good results if the enemy, as was the case in 1787, is not decidedly very superior. For although such an attack must be executed by means of dykes, and on that account cannot certainly have the advantage of much freedom of movement or of any great impulsive force, nevertheless, it is impossible for the offensive side to occupy all the dykes and roads which he does not require for his own purposes, and therefore the defensive with his better knowledge of the country, and being in possession of the strong points, should be able by some of the unoccupied dykes to effect a real flank attack against the columns of the assailant, or to cut them off from their sources of supply. If now, on the other hand, we reflect for a moment on the constrained position in which the assailant is placed, how much more dependent he is on his communications than in almost any other conceivable case, we may well imagine that every sally on the part of the defensive side which has the remotest possibility of success must at once as a demonstration be most effective. We doubt very much if the prudent and cautious duke of Brunswick would have ventured to approach Amsterdam if the Dutch had only made such a demonstration, from Utrecht for instance. About HackerNoon Book Series: We bring you the most important technical, scientific, and insightful public domain books. This book is part of the public domain. Carl von Clausewitz (2006). On War. Urbana, Illinois: Project Gutenberg. Retrieved https://www.gutenberg.org/cache/epub/1946/pg1946-images.html This eBook is for the use of anyone anywhere at no cost and with almost no restrictions whatsoever. You may copy it, give it away or re-use it under the terms of the Project Gutenberg License included with this eBook or online at www.gutenberg.org , located at https://www.gutenberg.org/policy/license.html .

The Inundations

After this long digression

THE VICAR OF BULLHAMPTON—SIR HARRY HOTSPUR—AN EDITOR'S TALES—CÆSAR.

St. Ewold's Parsonage

has always been about solving problems. Software development But the variation in HOW we solve the problems can be immense! Making this more complex is the one particular angle that our friends outside of development love to throw into the mix: "Buy vs. Build." Should we develop a solution in-house or purchase an existing one off the shelf? Understanding the Debate The buy vs. build decision is a simple one in essence: Do you create a solution yourself (build) or leverage an existing one (buy)? While the question may be straightforward, the answer is often intricate and relies on a myriad of factors -- some of which we in engineering aren't as sensitive to. Competitive Advantage: The Heart of the Matter A core concept to grasp in this discussion is the idea of competitive advantage. At its most basic, a competitive advantage is something a company can do uniquely well in comparison to its competitors, with a view to delivering better value to customers or outperforming its rivals. But why should developers care about competitive advantage? Every line of code written costs time and resources now and in the future. When you're developing something that isn't central to your competitive advantage, you're diverting those resources away from areas where they could be better spent to drive the underlying business forward. Consider these questions: Does this component provide a distinctive advantage over competitors? Is it Core? Is this a component found in most applications and doesn't distinguish our product? Is it Common? If something is core and not common, it's more likely something you should build. Anything that is generic and doesn't set your product or company apart—like user authentication or payment gateways—usually better to buy. Factors to Consider: Going Deeper 1. Time to Market Buying often accelerates your time to market. When you purchase a solution, especially well-established ones, you are essentially leveraging years of development, refinement, and testing that has already taken place. 2. Cost Implications Building usually has a higher initial AND ongoing cost than buying. However, it's essential to consider the total cost of ownership. With buying, licensing fees, subscription costs, and potential customization needs might add up over time. Weigh this carefully against the and equipment that must be maintained to support a build decision. A $5/mo monthly fee $150k/annual build decision might not balance against a $5k/mo buy decision. people plus 3. Customization & Flexibility Building provides more customization and flexibility, as you're creating a solution tailored to your exact requirements. However, with this flexibility comes the responsibility of maintenance. Of course, even if you find the exact functionality you're looking for in a buy decision, the lack of control may mean your desired use case shifts out of focus in the external organization you to change providers (or urgently build replacement functionality). forcing 4. Integration with Existing Systems Off-the-shelf solutions might not integrate seamlessly with your existing infrastructure or tools. Building in-house ensures that the solution fits like a glove with your current systems. This is a very common defense that tends to work well in conversations with folks outside of technology. Trouble is, while it might win the negotiation, this typically wins low-value low-leverage work for technology that can bogs down the group and may slow down the organization's broader pace of innovation. Be careful what you wish for. 5. Skill Set & Resources Does your team have the skills required to build the solution? If not, it might be more cost-effective to buy, rather than invest in training or hiring new talent. 6. Risk Assessment There's always a potential danger of vendor lock-in, where you become heavily dependent on an external solution that might not evolve as per your future needs or could become expensive down the line. While developing in-house might seem control-centric, there's the lurking risk of accumulating technical debt if not executed properly, which can become costly in terms of future maintenance and scalability. 7. Scalability Considerations Off-the-shelf solutions might cater to your current scale but might not adapt well when you expand or evolve, leading to potential inefficiencies. That said, constructing a bespoke solution allows you to keep scalability (and the other -ilities) in mind from the start, ensuring that the software grows with your business needs. However, unforeseen growth can sometimes challenge even the best in-house designs. 8. Support & Documentation A significant benefit is the availability of dedicated support teams and comprehensive documentation. Yet, the quality can vary, and relying on external support can sometimes introduce delays. But, an in-house solution gives you the onus of documenting the process and training developers. While this ensures tailored support and documentation, it requires regular updates and a dedicated onboarding process for new team members. In Conclusion: A Balancing Act The buy vs. build debate is not about finding a one-size-fits-all answer, but about understanding the nuances of your situation. For developers, it's about aligning technical capabilities with business strategy. We need to be wary of our biases. The ultimate goal is not just to solve a problem, but to solve it in a way that positions us and the company for success. Also published here.

Buy vs. Build: Navigating the Classic Dilemma for Developers

Unlocking the Future of DeFi: How Dolomite is Redefining the Crypto Landscape!

Desbloqueando el futuro de DeFi: ¡Cómo Dolomite está redefiniendo el paisaje criptográfico!

Déverrouiller l'avenir de DeFi : comment Dolomite redéfinit le paysage de la crypto !

DeFi के भविष्य को अनलॉक करना: कैसे डोलोमाइट क्रिप्टो लैंडस्केप को फिर से परिभाषित कर रहा है!

DeFi の未来を解き放つ: ドロマイトが仮想通貨の世界をどのように再定義しているか!

Desvendando o futuro do DeFi: como a Dolomite está redefinindo o cenário criptográfico!

Mở khóa tương lai của DeFi: Dolomite đang xác định lại bối cảnh tiền điện tử như thế nào!

解锁 DeFi 的未来：Dolomite 如何重新定义加密货币格局！

Intro In this article I’ll try to tell you about static analyzers and what they can bring to the code, to share with you possible difficulties, and to show you how the developer and analyzer work together. For my examples, I’ll use PHP, although solutions such as have implementations in many other languages as well. SonarQube So, let’s begin. What Are Analyzers For? Code analysis is an integral part of any developing process, on an equal level with testing and code reviews. Frequently, developers review the same code tens of times, checking that everything written fits the standards of code writing in the project, in the team, and in the company. In order to offload developers and speed up manual code analysis, various code analyzers were created. If the analyzer is set up correctly, the development process is accelerated, and the code quality increases. Static code analysis allows you to view the entire application code at once without the developer's participation which makes it different from manual or dynamic analysis. Static analysis, as the name says, does not run the code, but only looks through it for errors in the code, such as incorrectly specified types in function arguments, unused variables, etc. Static code analysis is performed by any team, even when developing alone. When writing code, every developer examines it for possible problems and smells. Because of this, a developer may make a mistake in the business logic when checking additional, technical parts of the work. Solutions for Static Code Analysis in PHP There are a lot of static analyzers available for PHP at the moment. Today, we will go through them with a short review of each, and then compare them so that you can choose the right one for your project. Short Review PHPStan Founded in 2016, it is one of the most popular static analyzers in PHP. At the moment, the project is supported by several developers, and the number of downloads of the package has exceeded 127 million. Among the main differences of PHPStan is the built-in support for plugins (available for example to work with modern frameworks). Has the ability to specify the level of checks. Does not have error types which is sometimes inconvenient when you need to enable or disable a particular one. Also has a paid GUI version of the application instead of CLI, which is important for some people. Phan In development since 2015, Phan has over 5 million downloads. Phan is much less popular but has serious weight as Rasmus Lerdorf, one of the creators of PHP, was also involved in its development. Among the advantages of Phan are its ability to act as a Language Server and real static. Phan does not attach the files it needs. It first builds the project tree using the PHP-ast library and then analyzes the files it needs. This method allows for faster analysis but has drawbacks. For example, Phan cannot count PHPDoc blocks inside methods, which often leads to false positives. Psalm Founded by Matt Brown of Vimeo in 2014, Psalm is also one of the top solutions for PHP. The project currently has over 33 million downloads. Can mention logic assertion checks and security hole checks as key features. Can perform some simple edits by itself. It also has an advantage over PHPStan, as it does not load unused files when checking part of the code base. On the plus side, it has excellent documentation where you can find all types of errors, as well as play in the sandbox and send to a colleague if necessary. It also has its own set of custom PHPDoc blocks for better analysis, e.g., type analysis. Mess Detector Created in 2005, Mess Detector has already been downloaded more than 65 million times. Mess Detector differs from all the analyzers mentioned before in that it analyzes code from the point of view of finding code with smells or without clean code principles. With its help, we can find out which classes we need to refactor because they have a lot of code/logical operations, or we can also see places not following design principles and clean code. PHP_CodeSniffer Its first version was released back in 2003, which puts it among the oldest static analyzers in PHP. Despite this, it is still being updated and has been downloaded more than 193 million times so far. The main difference with CodeSniffer is that it checks code for code style, which reduces its capabilities but gives it the speed of execution and the ability to fix many errors. Comparison Since all analyzers have a lot of possibilities, it is sometimes difficult to make a choice among the described solutions. Let's try to arrange the data by analyzers with the help of the table: Feature PhpStan Psalm Phan Mess Detector CodeSniffer Sandbox + + + - - CodeStyle - - - - + Variable Types + + + - + Method return types + + + - - Language Server extension + plugin - - IDE Integration + + w/o PHPStorm + + Plugins + + + - - Speed Fast, speed decreases as the complexity of the project increases Fast Medium Fast Fast Autofixer + Psalter + - + Method/Class size check - - - + - Сheck for unenforceable conditions + + + - - PHPDoc check + + +/- - + Union types check + + + - + Array shapes support + + - - - Possible Problems During Implementation The main problems during implementation are developers' unwillingness to get used to a new tool and incorrect initial settings. Due to the fact that when setting a more strict level of checks, the static analyzer often generates a huge number of errors, which greatly affects the psychological state of the team. Each static analyzer has 3 tools to solve this problem: - indicates the level of severity of startup checks. The easiest way to reduce the number of startup errors. After adding startup to the standard development process, you can gradually increase the level. Level - writes errors during the check to a file so that the analyzer does not generate errors that are already in the file during the next runs. With the help of this tool, you can remove old errors at once and keep an eye on not adding new ones. You can also create a task pool to gradually get rid of errors from Baseline. Baseline - helps to check only the code that has been modified by the developer. Due to the use of links, namespaces, and autoloaders, it is not always a preferable option because it cannot check the work of the whole application. Diff However, the speed of the analyzer's work in this case motivates the developer to run it more often. It is not worth using it in CI because you need to make sure that the whole application works correctly before sending it to the extension. Examples To test PHPStan, Phan, and Psalm, I used a large file with more than 700 lines. After the test launch of all three analyzers, it turned out that there were only 11 common places in the code where all three analyzers produced a Warning or Error. Let's go through the examples where some analyzers worked and others did not. PHPStan function getData(): ?Data;
function setData(Data $data): void;
$data = $user->getData(); 
$anotherUser->setData($data);
//PHPStan  Cannot call method setData() on \User|null. public function getPaginatedList(array $ids, int $page, int $pageSize): PaginatedList
// PHPStan Method App\Service\Service::getPaginatedList() has parameter // $ids with no value type specified in iterable type array.          
// 💡 See:  //https://phpstan.org/blog/solving-phpstan-no-value-type-specified-in-it//erable-type /** @var AnotherService */
private $anotherService;
private $logger;
//PHPStan Property App\Service\Service::$logger has no type specified Phan if (!empty($data['user']['address'])) {
    $this->geoService->searchByStreet($data['user']['address']);
}
//Phan: PhanTypeMismatchArgumentNullable Argument 1 ($address) is $data['user']['address'] of type ?false|?string but \App\Service\GeoService::searchByStreet() takes string defined at src/Service/GeoService.php:35 (expected type to be non-nullable) /** @var EntityManagerInterface */
private $entityManager;

public function __construct(
    EntityManagerInterface $entityManager
) {
    $this->entityManager = $entityManager;    
}

$repository = $this->entityManager->getRepository(User::class);
//Phan: PhanUndeclaredMethod Call to undeclared method \Doctrine\ORM\EntityManagerInterface::getRepository Psalm $doc = new DocumentDto(
$data->getDocument()['series'],
$data->getDocument()['number'],
new \DateTimeImmutable(
    $data->getDocument()['issuing_date']
));
//Psalm: INFO: PossiblyNullArgument - src/Service/Service.php:301:20 - Argument 2 of App\Dto\DocumentDto::__construct cannot be null, possibly null value provided (see https://psalm.dev/078)
//  $data->getDocument()['number'], public function doSomething(): ?string;

public function upsertAndDoSomething(
    int $id,
    Info $info
): string {
    return $this->doSomething();
}
//INFO: InvalidNullableReturnType - src/Service/Service.php:113:8 - The declared return type 'string' for App\Service\Service::upsertAndDoSomething is not nullable, but 'null|string' contains null (see https://psalm.dev/144)
): string { PHPMD Clean Code FILE: /app/src/Validator/Validator.php
------------------------------------------------------------------------
 42   | VIOLATION | Avoid using static access to class 'Enum' in method 'validate'.
 55   | VIOLATION | Avoid assigning values to variables in if clauses and the like (line '61', column '13').
 55   | VIOLATION | Avoid assigning values to variables in if clauses and the like (line '66', column '15').
 67   | VIOLATION | Avoid using static access to class '\App\Helpers\Helper' in method 'map'.
 70   | VIOLATION | Avoid using static access to class '\App\Requests\Request' in method 'map'. Design FILE: /app/src/Service/Service.php
-----------------------------------------------
 30 | VIOLATION | The class Service has a coupling between objects value of 24. Consider to reduce the number of dependencies under 13. Unused Code FILE: /app/src/Service/Service.php
------------------------------------------------
 31  | VIOLATION | Avoid unused parameters such as '$token'.
 32  | VIOLATION | Avoid unused parameters such as '$user'.
 33  | VIOLATION | Avoid unused parameters such as '$url'. Codesize FILE: /app/src/Consumer/Consumer.php
----------------------------------------------
 63   | VIOLATION | The method __construct has 10 parameters. Consider reducing the number of parameters to less than 10.
 92   | VIOLATION | The method execute() has a Cyclomatic Complexity of 22. The configured cyclomatic complexity threshold is 10.
 92   | VIOLATION | The method execute() has an NPath complexity of 6401. The configured NPath complexity threshold is 200.
 92   | VIOLATION | The method execute() has 124 lines of code. Current threshold is set to 100. Avoid really long methods. PHPCS FILE: src/Service/Service.php
------------------------------------------------------------------------------------------------------------------------------
FOUND 15 ERRORS AND 9 WARNINGS AFFECTING 22 LINES
------------------------------------------------------------------------------------------------------------------------------
 144 | WARNING | [ ] Line exceeds 120 characters; contains 151 characters
 184 | WARNING | [ ] Line exceeds 120 characters; contains 128 characters
 328 | ERROR   | [x] The first expression of a multi-line control structure must be on the line after the opening parenthesis
 336 | WARNING | [ ] Line exceeds 120 characters; contains 130 characters
 366 | WARNING | [ ] Line exceeds 120 characters; contains 127 characters
 395 | ERROR   | [x] Empty lines are not allowed in multi-line function calls
 440 | ERROR   | [x] Opening brace must be the last content on the line
 440 | ERROR   | [x] Closing brace must be on a line by itself
 440 | ERROR   | [x] Each PHP statement must be on a line by itself
 481 | WARNING | [ ] Line exceeds 120 characters; contains 149 characters
 486 | ERROR   | [x] The first expression of a multi-line control structure must be on the line after the opening parenthesis
 488 | WARNING | [ ] Line exceeds 120 characters; contains 122 characters
 503 | ERROR   | [x] Opening parenthesis of a multi-line function call must be the last content on the line
 545 | WARNING | [ ] Line exceeds 120 characters; contains 122 characters
 546 | WARNING | [ ] Line exceeds 120 characters; contains 138 characters
 587 | WARNING | [ ] Line exceeds 120 characters; contains 144 characters
 588 | ERROR   | [x] The first expression of a multi-line control structure must be on the line after the opening parenthesis
 594 | ERROR   | [x] The first expression of a multi-line control structure must be on the line after the opening parenthesis
 602 | ERROR   | [x] The first expression of a multi-line control structure must be on the line after the opening parenthesis
 619 | ERROR   | [x] The first expression of a multi-line control structure must be on the line after the opening parenthesis
 626 | ERROR   | [x] The first expression of a multi-line control structure must be on the line after the opening parenthesis
 634 | ERROR   | [x] The first expression of a multi-line control structure must be on the line after the opening parenthesis
 679 | ERROR   | [x] The first expression of a multi-line control structure must be on the line after the opening parenthesis
 724 | ERROR   | [x] Expected 1 space after closing brace; newline found
------------------------------------------------------------------------------------------------------------------------------
PHPCBF CAN FIX THE 15 MARKED SNIFF VIOLATIONS AUTOMATICALLY
------------------------------------------------------------------------------------------------------------------------------ Collective Use As you can see in the examples - at the moment, every analyzer cannot meet all the desires and requirements of code coverage. That's why many teams that use such tools use several tools instead of one. It helps to cover most of the errors and also to look at one and the same error from different sides, which makes it possible to fix it faster. Conclusions Despite the fact that static analyzers appeared quite a long time ago, due to the specifics of their implementation, they perform their work differently and have different results too. Each development team can choose the tool it needs. In my opinion, using PHP_CodeSniffer is mandatory for all projects, and as for the choice between PHPStan/Phan/Psalm, we decided to use two of them in our project - Psalm and Phan because Psalm in our realities showed similar results to PHPStan, but has typed errors, which allows us to easily integrate the result with other services, such as Slack or Jira.

Stories, News & Mentions about NOTCOMMON

Notcommon’s Stories on HackerNoon