Ruffo

ruffo

This paper is available on arxiv under CC 4.0 license. Authors: (1) Jeffrey Ouyang-Zhang, UT Austin (2) Daniel J. Diaz, UT Austin (3) Adam R. Klivans, UT Austin (4) Philipp Krähenbühl, UT Austin Table of Links Abstract & Introduction Related Works Preliminary Methodology Results Limitations Conclusion, Acknowledgements, & References Experimental Setting & Additional Results 7 Conclusion We present a method that efficiently scales thermodynamic stability prediction from single mutations to higher-order mutations. Our key insight is that the effects of mutations on the same protein are correlated. Thus, for a target protein, it suffices to run a deep backbone once and decode the effect of all mutations simultaneously using a shallow decoder. With the AlphaFold model as our backbone, our method outperforms existing methods on a variety of single and multiple mutation benchmarks. Our method scales to millions of mutations with minimal computational overhead and runs in a fraction of the time it would take prior works. 8 Acknowledgements This work is supported by the NSF AI Institute for Foundations of Machine Learning (IFML). References [1] Randomization of genes by pcr mutagenesis. Genome research 2(1), 28–33 (1992) 2 [2] Adams, J.P., Brown, M.J., Diaz-Rodriguez, A., Lloyd, R.C., Roiban, G.D.: Biocatalysis: A Pharma Perspective. Advanced Synthesis and Catalysis 361(11), 2421–2432 (2019). https://doi.org/10.1002/adsc.201900424 1 [3] Ahdritz, G., Bouatta, N., Kadyan, S., Xia, Q., Gerecke, W., O’Donnell, T.J., Berenberg, D., Fisk, I., Zanichelli, N., Zhang, B., Nowaczynski, A., Wang, B., Stepniewska-Dziubinska, M.M., Zhang, S., Ojewole, A., Guney, M.E., Biderman, S., Watkins, A.M., Ra, S., Lorenzo, P.R., Nivon, L., Weitzner, B., Ban, Y.E.A., Sorger, P.K., Mostaque, E., Zhang, Z., Bonneau, R., AlQuraishi, M.: Openfold: Retraining alphafold2 yields new insights into its learning mechanisms and capacity for generalization. bioRxiv (2022). https://doi.org/10.1101/2022.11.20.517210 6 [4] Arnold, F.H.: Design by directed evolution. Accounts of chemical research 31(3), 125–131 (1998) 1, 2, 3 [5] Bell, E.L., Finnigan, W., France, S.P., Green, A.P., Hayes, M.A., Hepworth, L.J., Lovelock, S.L., Niikura, H., Osuna, S., Romero, E., Ryan, K.S., Turner, N.J., Flitsch, S.L.: Biocatalysis. Nature Reviews Methods Primers 1(1), 1–21 (2021). https://doi.org/10.1038/s43586-021-00044-z 1 [6] Benevenuta, S., Pancotti, C., Fariselli, P., Birolo, G., Sanavia, T.: An antisymmetric neural network to predict free energy changes in protein variants. Journal of Physics D: Applied Physics 54(24), 245403 [7] Benevenuta, S., Pancotti, C., Fariselli, P., Birolo, G., Sanavia, T.: An antisymmetric neural network to predict free energy changes in protein variants. Journal of Physics D: Applied Physics 54(24), 245403 (2021) 8 [8] Benevenuta, S., Birolo, G., Sanavia, T., Capriotti, E., Fariselli, P.: Challenges in predicting stabilizing variations: An exploration. Frontiers in Molecular Biosciences 9, 1075570 (2023) 5, 6, 17 [9] Brückner, A., Polge, C., Lentze, N., Auerbach, D., Schlattner, U.: Yeast two-hybrid, a powerful tool for systems biology. International journal of molecular sciences 10(6), 2763–2788 (2009) 1 [10] Cadet, F., Saavedra, E., Syren, P.O., Gontero, B.: Machine learning, epistasis, and protein engineering: From sequence-structure-function relationships to regulation of metabolic pathways. Frontiers in Molecular Biosciences 9, 1098289 (2022) 1 [11] Capriotti, E., Fariselli, P., Casadio, R.: I-mutant2. 0: predicting stability changes upon mutation from the protein sequence or structure. Nucleic acids research 33(suppl_2), W306–W310 (2005) 3, 8 [12] Chen, K., Arnold, F.H.: Tuning the activity of an enzyme for unusual environments: sequential random mutagenesis of subtilisin e for catalysis in dimethylformamide. Proceedings of the National Academy of Sciences 90(12), 5618–5622 (1993) 1, 3 [13] Chen, T., Gong, C., Diaz, D.J., Chen, X., Wells, J.T., Wang, Z., Ellington, A., Dimakis, A., Klivans, A., et al.: Hotprotein: A novel framework for protein thermostability prediction and editing. In: The Eleventh International Conference on Learning Representations (2022) 3 [14] Chen, Y., Lu, H., Zhang, N., Zhu, Z., Wang, S., Li, M.: Premps: Predicting the impact of missense mutations on protein stability. PLoS computational biology 16(12), e1008543 (2020) 3, 8 [15] Cheng, J., Randall, A., Baldi, P.: Prediction of protein stability changes for single-site mutations using support vector machines. Proteins: Structure, Function, and Bioinformatics 62(4), 1125–1132 (2006) 3, 8 [16] Dehghanpoor, R., Ricks, E., Hursh, K., Gunderson, S., Farhoodi, R., Haspel, N., Hutchinson, B., Jagodzinski, F.: Predicting the effect of single and multiple mutations on protein structural stability. Molecules 23(2), 251 (2018) 4, 7 [17] Dehouck, Y., Kwasigroch, J.M., Gilis, D., Rooman, M.: Popmusic 2.1: a web server for the estimation of protein stability changes upon mutation and sequence optimality. BMC bioinformatics 12(1), 1–12 (2011) 3, 8 [18] Diaz, D.J., Gong, C., Ouyang-Zhang, J., Loy, J.M., Wells, J.T., Yang, D., Ellington, A.J., Dimakis, A., Klivans, A.R.: Stability oracle: A structure-based graph-transformer for identifying stabilizing mutations (2023). https://doi.org/10.1101/2023.05.15.540857 3, 4, 7, 8, 16, 17, 18, 19 [19] Diaz, D.J., Kulikova, A.V., Ellington, A.D., Wilke, C.O.: Using machine learning to predict the effects and consequences of mutations in proteins. Current Opinion in Structural Biology 78, 102518 (2023) 3 [20] d’Oelsnitz, S., Diaz, D.J., Acosta, D.J., Schechter, M.W., Minus, M.B., Howard, J.R., Do, H., Loy, J., Alper, H., Ellington, A.D.: Synthetic microbial sensing and biosynthesis of amaryllidaceae alkaloids. bioRxiv pp. 2023–04 (2023) 3 [21] d’Oelsnitz, S., Kim, W., Burkholder, N.T., Javanmardi, K., Thyer, R., Zhang, Y., Alper, H.S., Ellington, A.D.: Using fungible biosensors to evolve improved alkaloid biosyntheses. Nature Chemical Biology 18(9), 981–989 (2022) 1 [22] Ellefson, J.W., Gollihar, J., Shroff, R., Shivram, H., Iyer, V.R., Ellington, A.D.: Synthetic evolutionary origin of a proofreading reverse transcriptase. Science 352(6293), 1590–1593 (2016) 1 [23] Feng, X., Sanchis, J., Reetz, M.T., Rabitz, H.: Enhancing the efficiency of directed evolution in focused enzyme libraries by the adaptive substituent reordering algorithm. Chemistry–A European Journal 18(18), 5646–5654 (2012) 3 [24] Frazer, J., Notin, P., Dias, M., Gomez, A., Min, J.K., Brock, K., Gal, Y., Marks, D.S.: Disease variant prediction with deep generative models of evolutionary data. Nature 599(7883), 91–95 (2021) 9 [25] Gebauer, M., Skerra, A.: Engineered protein scaffolds as next-generation therapeutics. Annual Review of Pharmacology and Toxicology 60, 391–415 (2020). https://doi.org/10.1146/annurev-pharmtox-010818- 021118 1 [26] Gerasimavicius, L., Livesey, B.J., Marsh, J.A.: Correspondence between functional scores from deep mutational scans and predicted effects on protein stability. bioRxiv pp. 2023–02 (2023) 9 [27] Giver, L., Gershenson, A., Freskgard, P.O., Arnold, F.H.: Directed evolution of a thermostable esterase. Proceedings of the National Academy of Sciences 95(22), 12809–12813 (1998) 2, 3 [28] Hie, B.L., Shanker, V.R., Xu, D., Bruun, T.U., Weidenbacher, P.A., Tang, S., Wu, W., Pak, J.E., Kim, P.S.: Efficient evolution of human antibodies from general protein language models. Nature Biotechnology (2023) 3 [29] Jaroszewicz, W., Morcinek-Orłowska, J., Pierzynowska, K., Gaffke, L., W˛egrzyn, G.: Phage display and other peptide display technologies. FEMS Microbiology Reviews 46(2), fuab052 (2022) 1 [30] Järvelin, K., Kekäläinen, J.: Cumulated gain-based evaluation of ir techniques. ACM Transactions on Information Systems (TOIS) 20(4), 422–446 (2002) 6, 17 [31] Jumper, J., Evans, R., Pritzel, A., Green, T., Figurnov, M., Ronneberger, O., Tunyasuvunakool, K., Bates, R., Žídek, A., Potapenko, A., et al.: Highly accurate protein structure prediction with alphafold. Nature 596(7873), 583–589 (2021) 2, 3, 4, 6, 9, 10 [32] Kellogg, E.H., Leaver-Fay, A., Baker, D.: Role of conformational sampling in computing mutation-induced changes in protein structure and stability. Proteins: Structure, Function, and Bioinformatics 79(3), 830–838 (2011) 3 [33] Klabunde, T., Petrassi, H.M., Oza, V.B., Raman, P., Kelly, J.W., Sacchettini, J.C.: Rational design of potent human transthyretin amyloid disease inhibitors. Nature structural biology 7(4), 312–321 (2000) 1, 2 [34] Kouba, P., Kohout, P., Haddadi, F., Bushuiev, A., Samusevich, R., Sedlar, J., Damborsky, J., Pluskal, T., Sivic, J., Mazurenko, S.: Machine learning-guided protein engineering. ACS Catalysis 13, 13863–13895 (2023) 3 [35] Laimer, J., Hofer, H., Fritz, M., Wegenkittl, S., Lackner, P.: Maestro-multi agent stability prediction upon point mutations. BMC bioinformatics 16(1), 1–13 (2015) 3, 4, 7, 8 [36] Li, B., Yang, Y.T., Capra, J.A., Gerstein, M.B.: Predicting changes in protein thermodynamic stability upon point mutation with deep 3d convolutional neural networks. PLoS computational biology 16(11), e1008291 (2020) 3, 4, 8 [37] Li, G., Panday, S.K., Alexov, E.: Saafec-seq: a sequence-based method for predicting the effect of single point mutations on protein thermodynamic stability. International journal of molecular sciences 22(2), 606 (2021) 3 [38] Lin, Z., Akin, H., Rao, R., Hie, B., Zhu, Z., Lu, W., Smetanin, N., dos Santos Costa, A., Fazel-Zarandi, M., Sercu, T., Candido, S., et al.: Language models of protein sequences at the scale of evolution enable accurate structure prediction. bioRxiv (2022) 3, 4, 6, 9, 10, 18 [39] Loell, K., Nanda, V.: Marginal protein stability drives subcellular proteome isoelectric point. Proceedings of the National Academy of Sciences 115(46), 11778–11783 (2018) 1 [40] Lu, H., Diaz, D.J., Czarnecki, N.J., Zhu, C., Kim, W., Shroff, R., Acosta, D.J., Alexander, B.R., Cole, H.O., Zhang, Y., et al.: Machine learning-aided engineering of hydrolases for pet depolymerization. Nature 604(7907), 662–667 (2022) 3 [41] Meghwanshi, G.K., Kaur, N., Verma, S., Dabi, N.K., Vashishtha, A., Charan, P.D., Purohit, P., Bhandari, H.S., Bhojak, N., Kumar, R.: Enzymes for pharmaceutical and therapeutic applications. Biotechnology and Applied Biochemistry 67(4), 586–601 (2020). https://doi.org/10.1002/bab.1919 1 [42] Meier, J., Rao, R., Verkuil, R., Liu, J., Sercu, T., Rives, A.: Language models enable zero-shot prediction of the effects of mutations on protein function. Advances in Neural Information Processing Systems 34, 29287–29303 (2021) 9, 18 [43] Mirdita, M., Schütze, K., Moriwaki, Y., Heo, L., Ovchinnikov, S., Steinegger, M.: Colabfold: making protein folding accessible to all. Nature methods 19(6), 679–682 (2022) 6 [44] Montanucci, L., Capriotti, E., Frank, Y., Ben-Tal, N., Fariselli, P.: Ddgun: an untrained method for the prediction of protein stability changes upon single and multiple point variations. BMC bioinformatics 20, 1–10 (2019) 3, 4, 7, 8, 16, 18 [45] Montanucci, L., Savojardo, C., Martelli, P.L., Casadio, R., Fariselli, P.: On the biases in predictions of protein stability changes upon variations: the inps test case. Bioinformatics 35(14), 2525–2527 (2019) 16 [46] Musdal, Y., Govindarajan, S., Mannervik, B.: Exploring sequence-function space of a poplar glutathione transferase using designed information-rich gene variants. Protein Engineering, Design and Selection 30(8), 543–549 (2017) 3 [47] Nijkamp, E., Ruffolo, J., Weinstein, E.N., Naik, N., Madani, A.: Progen2: exploring the boundaries of protein language models. arXiv preprint arXiv:2206.13517 (2022) 9 [48] Nikam, R., Kulandaisamy, A., Harini, K., Sharma, D., Gromiha, M.M.: Prothermdb: thermodynamic database for proteins and mutants revisited after 15 years. Nucleic acids research 49(D1), D420–D424 (2021) 2, 16 [49] Notin, P., Dias, M., Frazer, J., Hurtado, J.M., Gomez, A.N., Marks, D., Gal, Y.: Tranception: protein fitness prediction with autoregressive transformers and inference-time retrieval. In: International Conference on Machine Learning. pp. 16990–17017. PMLR (2022) 2, 3, 9, 16, 18 [50] Notin, P.M., Van Niekerk, L., Kollasch, A.W., Ritter, D., Gal, Y., Marks, D.: Trancepteve: Combining family-specific and family-agnostic models of protein sequences for improved fitness prediction. bioRxiv pp. 2022–12 (2022) 9 [51] Nutschel, C., Fulton, A., Zimmermann, O., Schwaneberg, U., Jaeger, K.E., Gohlke, H.: Systematically scrutinizing the impact of substitution sites on thermostability and detergent tolerance for bacillus subtilis lipase a. Journal of chemical information and modeling 60(3), 1568–1584 (2020) 9 [52] Paik, I., Ngo, P.H., Shroff, R., Diaz, D.J., Maranhao, A.C., Walker, D.J., Bhadra, S., Ellington, A.D.: Improved bst dna polymerase variants derived via a machine learning approach. Biochemistry (2021) 3 [53] Pak, M.A., Markhieva, K.A., Novikova, M.S., Petrov, D.S., Vorobyev, I.S., Maksimova, E.S., Kondrashov, F.A., Ivankov, D.N.: Using alphafold to predict the impact of single mutations on protein stability and function. Plos one 18(3), e0282689 (2023) 3 [54] Pancotti, C., Benevenuta, S., Birolo, G., Alberini, V., Repetto, V., Sanavia, T., Capriotti, E., Fariselli, P.: Predicting protein stability changes upon single-point mutation: a thorough comparison of the available tools on a new dataset. Briefings in Bioinformatics 23(2), bbab555 (2022) 2, 8, 16, 17 [55] Pancotti, C., Benevenuta, S., Repetto, V., Birolo, G., Capriotti, E., Sanavia, T., Fariselli, P.: A deep-learning sequence-based method to predict protein stability changes upon genetic variations. Genes 12(6), 911 (2021) 8 [56] Pires, D.E., Ascher, D.B., Blundell, T.L.: Duet: a server for predicting effects of mutations on protein stability using an integrated computational approach. Nucleic acids research 42(W1), W314–W319 (2014) 3, 8 [57] Pires, D.E., Ascher, D.B., Blundell, T.L.: mcsm: predicting the effects of mutations in proteins using graph-based signatures. Bioinformatics 30(3), 335–342 (2014) 8 [58] Pucci, F., Schwersensky, M., Rooman, M.: Artificial intelligence challenges for predicting the impact of mutations on protein stability. Current opinion in structural biology 72, 161–168 (2022) 17 [59] Qiu, Y., Wei, G.W.: Persistent spectral theory-guided protein engineering. Nature Computational Science pp. 1–15 (2023) 3, 6, 17 [60] Rao, R.M., Liu, J., Verkuil, R., Meier, J., Canny, J., Abbeel, P., Sercu, T., Rives, A.: Msa transformer. In: International Conference on Machine Learning. pp. 8844–8856. PMLR (2021) 3, 9, 10 [61] Riesselman, A.J., Ingraham, J.B., Marks, D.S.: Deep generative models of genetic variation capture the effects of mutations. Nature methods 15(10), 816–822 (2018) 9 [62] Rives, A., Meier, J., Sercu, T., Goyal, S., Lin, Z., Liu, J., Guo, D., Ott, M., Zitnick, C.L., Ma, J., Fergus, R.: Biological structure and function emerge from scaling unsupervised learning to 250 million protein sequences. PNAS (2019). https://doi.org/10.1101/622803 3, 7 [63] Rodrigues, C.H., Pires, D.E., Ascher, D.B.: Dynamut: predicting the impact of mutations on protein conformation, flexibility and stability. Nucleic acids research 46(W1), W350–W355 (2018) 3, 8 [64] Rodrigues, C.H., Pires, D.E., Ascher, D.B.: Dynamut2: Assessing changes in stability and flexibility upon single and multiple point missense mutations. Protein Science 30(1), 60–69 (2021) 3, 4, 7 [65] Sanavia, T., Birolo, G., Montanucci, L., Turina, P., Capriotti, E., Fariselli, P.: Limitations and challenges in protein stability prediction upon genome variations: towards future applications in precision medicine. Computational and Structural Biotechnology Journal 18, 1968–1979 (2020). https://doi.org/10.1016/j.csbj.2020.07.011, https://doi.org/10.1016/j.csbj.2020.07.011 6, 17 [66] Savojardo, C., Fariselli, P., Martelli, P.L., Casadio, R.: Inps-md: a web server to predict stability of protein variants from sequence and structure. Bioinformatics 32(16), 2542–2544 (2016) 3, 8 [67] Schymkowitz, J., Borg, J., Stricher, F., Nys, R., Rousseau, F., Serrano, L.: The foldx web server: an online force field. Nucleic acids research 33(suppl_2), W382–W388 (2005) 3, 7, 8, 18 [68] Shroff, R., Cole, A.W., Diaz, D.J., Morrow, B.R., Donnell, I., Annapareddy, A., Gollihar, J., Ellington, A.D., Thyer, R.: Discovery of novel gain-of-function mutations guided by structure-based deep learning. ACS synthetic biology 9(11), 2927–2935 (2020) 3 [69] Starr, T.N., Thornton, J.W.: Epistasis in protein evolution. Protein science 25(7), 1204–1218 (2016) 1 [70] Steinegger, M., Soding, J.: Mmseqs2 enables sensitive protein sequence searching for the analysis of massive data sets. Nature biotechnology 35(11), 1026–1028 (2017) 6 [71] Stemmer, W.P.: Rapid evolution of a protein in vitro by dna shuffling. Nature 370(6488), 389–391 (1994) 2, 3 [72] Tsuboyama, K., Dauparas, J., Chen, J., Laine, E., Mohseni Behbahani, Y., Weinstein, J.J., Mangan, N.M., Ovchinnikov, S., Rocklin, G.J.: Mega-scale experimental analysis of protein folding stability in biology and protein design. bioRxiv pp. 2022–12 (2022) 1, 2, 6, 16 [73] Tung, J.W., Heydari, K., Tirouvanziam, R., Sahaf, B., Parks, D.R., Herzenberg, L.A., Herzenberg, L.A.: Modern flow cytometry: a practical approach. Clinics in laboratory medicine 27(3), 453–468 (2007) 1 [74] Umerenkov, D., Shashkova, T.I., Strashnov, P.V., Nikolaev, F., Sindeeva, M., Ivanisenko, N.V., Kardymon, O.L.: Prostata: Protein stability assessment using transformers. bioRxiv pp. 2022–12 (2022) 3, 4, 6, 7, 8, 18, 19 [75] Wittmann, B.J., Johnston, K.E., Wu, Z., Arnold, F.H.: Advances in machine learning for directed evolution. Current opinion in structural biology 69, 11–18 (2021) 3 [76] Wittmann, B.J., Yue, Y., Arnold, F.H.: Informed training set design enables efficient machine learningassisted directed protein evolution. Cell Systems 12(11), 1026–1045 (2021) 3 [77] Worth, C.L., Preissner, R., Blundell, T.L.: Sdm—a server for predicting effects of mutations on protein stability and malfunction. Nucleic acids research 39(suppl_2), W215–W222 (2011) 3, 8 [78] Wu, S., Snajdrova, R., Moore, J.C., Baldenius, K., Bornscheuer, U.T.: Biocatalysis: Enzymatic Synthesis for Industrial Applications. Angewandte Chemie - International Edition 60(1), 88–119 (2021). https://doi.org/10.1002/anie.202006648 1 [79] Yeung, N., Lin, Y.W., Gao, Y.G., Zhao, X., Russell, B.S., Lei, L., Miner, K.D., Robinson, H., Lu, Y.: Rational design of a structural and functional nitric oxide reductase. Nature 462(7276), 1079–1082 (2009) 1, 2 [80] Zhou, B., Lv, O., Yi, K., Xiong, X., Tan, P., Hong, L., Wang, Y.G.: Accurate and definite mutational effect prediction with lightweight equivariant graph neural networks. arXiv preprint arXiv:2304.08299 (2023) 3 [81] Zhou, Y., Pan, Q., Pires, D.E., Rodrigues, C.H., Ascher, D.B.: Ddmut: predicting effects of mutations on protein stability using deep learning. Nucleic Acids Research p. gkad472 (2023) 3

Predicting a Protein’s Stability
under a Million Mutations: Conclusion, Acknowledgement & References

The Future of Social Networking: User Empowerment, Privacy, and Freedom From Censorship

The inaugural is on the books. More than 56,000 registrants from all over the world, including those at live studios in four cities — Sydney, London, New York, and San Francisco. At any given time live viewers averaged more than 15,000. The purpose was to connect people from all over the world in a way that had never been done before. Global Marketing Day “We wanted to do something unique, something that nobody had ever done before and something that will connect people across the whole world,” says Chief Strategy and Corporate Development Officer . “It was a learning curve for the whole team, we had experience in making both online events and offline events, but not on this scale. We have learned a lot. It was probably one of the most viewed online events of the year in MarTech. Getting the same audience with any sort of offline event would be very hard.” SEMRush Eugine Levin Topics ranged from making videos for business to positioning yourself as a leading expert in your field to the importance of putting your customers first to repurposing social media content for other platforms. Basically any marketing topic you could think of, there was a unique twist put on it. Each session had its own show host. In New York, hosted expert speakers including and . We talked about live streaming, particularly the power of using it for your business and personal brands across networks, especially LinkedIn. ( ) Lee Odden Bob Ruffolo me video here The massive effort that went into pulling this off globally is obvious when you see the breadth of topic videos – . But will they do it again next year? available here “We might slightly change format to make it even better and bigger, but we will definitely do something,” says Levin. Here’s what other participants had to say: “It was great to participate in Global Marketing Day and share deep insights about Personal Branding with two of my marketing favorites,” said Brian Fanzo and Stephanie Liu. “We reviewed strong organic content strategy for building a personal brand that stands up and confidently is unique.” – Goldie Chan , LinkedIn Influencer and Founder of Warm Robots “Global Marketing Day was the best online conference that I have ever attended or spoken at. It involved a large number of marketers from across the globe, featuring a diverse array of topics under social media and digital marketing. Logistically, it almost seemed impossible; yet, SEMrush made everything run smoothly and beautifully without losing any personal touch. I gained valuable feedback and engagement from the live audience for my talk and our session on live streaming. Very few online conferences can have the level of audience engagement that Global Marketing Day received. A hearty congratulations to Anton Shulke and everyone who worked on this to make such a successful event.” – Dr. Ai Addyson-Zhang , Founder of Classroom Without Walls “SEMrush asked me to be the backup and step up to the plate if there was a no-show from a guest, host or speaker. There were no no-shows, so I spent the entire New York leg in the studio hanging out in the Green room (that wasn’t green), surrounded by photos of the stars (Julie Andrews, Paul Simon, Oprah Winfrey, Barack Obama). Turned out to be a unique experience: for over 7 hours, there was a steady stream of more than 40 digital marketing people coming through the green room on the way to and from the sessions. And that meant literally hundreds of phenomenally interesting top-drawer digital marketing conversations to join in or tune into. Brilliant. Luckily, 4 of them were recorded for posterity on the #SEOisAEO podcast!” – Jason Barnard , Keynote Speaker and Master of Ceremonies at Kalicube “The first global marketing day event was beyond inspiring. It’s amazing how they were able to assemble the all-stars of marketing for one power-packed event. The marketing world needed this event. I was grateful to be a part of it and I can’t wait for next year!” – Tim Salau , Mr. Future of Work Photos Courtesy of Courtesy of Scot and Susan Westwater

56k+ Marketers Joined Global Marketing Day 

A Storm Brewing: Barclays' Cost-Cutting Plan Ruffles Feathers Hey there! So, it seems like British bank Barclays is planning to make some big changes. British Barclays is stirring up a tempest with its reported plan to slash around 2,000 jobs. According to trusty sources at Reuters, the bank is on a mission to trim costs by a whopping £1 billion ($1.25 billion). Brace yourselves for the details that lie ahead! Cutting Deep: The Back-Office Shake-Up In the realm of profit-boosting, Barclays' managers are leaving no stone unturned. Whispers in the wind suggest that proposals are being reviewed, potentially targeting 1,500 to 2,000 jobs in back-office departments like legal and HR. Those roles, my friends, might just be facing the chopping block. Impact Zone: Barclays Execution Services in the Crosshairs The battlefield of job cuts is predicted to primarily affect Barclays Execution Services (BX), a mighty service company that supports various arms of the bank. Currently employing over a quarter of Barclays' troops, around 22,300 valiant souls, BX is bracing for the storm. A Twist in the Tale: Priority Plays Hide and Seek Ah, the unpredictable nature of it all! While discussions about the looming layoffs continue, whispers from the shadows suggest that the bank might end up shifting its focus to other areas. The plot thickens, my friends, leaving us guessing about the final targets of this restructuring saga. Weathering the Storm: Barclays' Q3 Results and the Quest for Efficiency A dark cloud looms over Barclays' recent Q3 results, revealing a dreary 16% drop in Q3 profits, amounting to £1.3bn. But fear not, for , stands firm. He vows to seek out inefficiencies, paving the way to lower "structural costs." CS Venkatakrishnan, the group chief executive of Barclays All eyes turn towards the investor presentation scheduled for February, eagerly awaiting further clues to the bank's strategy. Unraveling Fate: The Saga Continues And so, dear readers, we find ourselves in the midst of this gripping saga. The cards have been dealt, and the winds of change are blowing through Barclays. As we await the unveiling of their grand plan, let us hope for the best, keeping our thoughts with the employees who might face uncertain times.

Barclays' Cost-Cutting Plan Ruffles Feathers

Introduction This is a companion guide to (the "Part 1 guide"). I wrote the Part 1 guide on how to finetune Meta AI's model on Chilean Spanish. It is assumed that you have completed that guide and have generated your own finetuned XLS-R model. This guide will explain the steps to run on your finetuned XLS-R model via a . Working With wav2vec2 Part 1 - Finetuning XLS-R for Automatic Speech Recognition wav2vec2 XLS-R ("XLS-R") inference Kaggle Notebook Prerequisites and Before You Get Started To complete the guide, you will need to have: A finetuned XLS-R model for the Spanish language. An existing . Kaggle account Intermediate knowledge of Python. Intermediate knowledge of working with Kaggle Notebooks. Intermediate knowledge of ML concepts. Basic knowledge of ASR concepts. Building the Inference Notebook Step 1 - Setting Up Your Kaggle Environment Step 1.1 - Creating a New Kaggle Notebook Log in to Kaggle. Create a new Kaggle Notebook. The name of the notebook can be changed as desired. This guide uses the notebook name . spanish-asr-inference Step 1.2 - Adding the Test Datasets This guide uses the as its source for test data. Like the , the Peruvian speakers dataset also consists of two sub-datasets: 2,918 recordings of male Peruvian speakers and 2,529 recordings of female Peruvian speakers. Peruvian Spanish Speech Data Set Chilean Spanish Speech Data Set This dataset has been uploaded to Kaggle as 2 distinct datasets: Recordings of Peruvian male speakers Recordings of Peruvian female speakers Add both of these datasets to your Kaggle Notebook by clicking on . Add Input Step 1.3 - Adding the Finetuned Model You should have saved your finetuned model in of the guide as a . Step 4 Working With wav2vec2 Part 1 - Finetuning XLS-R for Automatic Speech Recognition Kaggle Model Add your finetuned model to your Kaggle Notebook by clicking on . Add Input Step 2 - Building the Inference Notebook The following 16 sub-steps build each of the inference notebook's 16 cells in order. You will note that many of the same utility methods from the Part 1 guide are used here. Step 2.1 - CELL 1: Installing Packages The first cell of the inference notebook installs dependencies. Set the first cell to: ### CELL 1: Install Packages ###
!pip install --upgrade torchaudio
!pip install jiwer Step 2.2 - CELL 2: Importing Python Packages The second cell imports required Python packages. Set the second cell to: ### CELL 2: Import Python packages ###
import re
import math
import random
import pandas as pd
import torchaudio
from datasets import load_metric
from transformers import pipeline Step 2.3 - CELL 3: Loading WER Metric The third cell imports the HuggingFace WER evaluation metric. Set the third cell to: ### CELL 3: Load WER metric ###
wer_metric = load_metric("wer") WER will be used to measure the performance of the finetuned model on test data. Step 2.4 - CELL 4: Setting Constants The fourth cell sets constants that will be used throughout the notebook. Set the fourth cell to: ### CELL 4: Constants ###
# Testing data
TEST_DATA_PATH_MALE = "/kaggle/input/google-spanish-speakers-peru-male/"
TEST_DATA_PATH_FEMALE = "/kaggle/input/google-spanish-speakers-peru-female/"
EXT = ".wav"
NUM_LOAD_FROM_EACH_SET = 3

# Special characters
SPECIAL_CHARS = r"[\d\,\-\;\!\¡\?\¿\।\‘\’\"\–\'\:\/\.\“\”\৷\…\‚\॥\\]"

# Sampling rates
ORIG_SAMPLING_RATE = 48000
TGT_SAMPLING_RATE = 16000 Step 2.5 - CELL 5: Utility Methods for Reading Index Files, Cleaning Text, and Creating Vocabulary The fifth cell defines utility methods for reading the dataset index files, as well as for cleaning transcription text and generating a random set of samples from test data. Set the fifth cell to: ### CELL 5: Utility methods for reading index files, cleaning text, random indices generator ###
def read_index_file_data(path: str, filename: str):
    data = []
    with open(path + filename, "r", encoding = "utf8") as f:
        lines = f.readlines()
        for line in lines:
            file_and_text = line.split("\t")
            data.append([path + file_and_text[0] + EXT, file_and_text[1].replace("\n", "")])
    return data

def clean_text(text: str) -> str:
    cleaned_text = re.sub(SPECIAL_CHARS, "", text)
    cleaned_text = cleaned_text.lower()
    return cleaned_text

def get_random_samples(dataset: list, num: int) -> list:
    used = []
    samples = []
    for i in range(num):
        a = -1
        while a == -1 or a in used:
            a = math.floor(len(dataset) * random.random())
        samples.append(dataset[a])
        used.append(a)
    return samples The method reads a dataset index file and produces a list of lists with audio filename and transcription data, e.g.: read_index_file_data line_index.tsv [
    ["/kaggle/input/google-spanish-speakers-chile-male/clm_08421_01719502739", "Es un viaje de negocios solamente voy por una noche"]
    ...
] The method is used to strip each text transcription of the characters specified by the regular expression assigned to in . These characters, inclusive of punctuation, can be eliminated as they don't provide any semantic value when training the model to learn mappings between audio features and text transcriptions. clean_text SPECIAL_CHARS Step 2.4 The method returns a set of random test samples with the quantity set by the constant in . get_random_samples NUM_LOAD_FROM_EACH_SET Step 2.4 Step 2.6 - CELL 6: Utility Methods for Loading and Resampling Audio Data The sixth cell defines utility methods using to load and resample audio data. Set the sixth cell to: torchaudio ### CELL 7: Utility methods for loading and resampling audio data ###
def read_audio_data(file):
    speech_array, sampling_rate = torchaudio.load(file, normalize = True)
    return speech_array, sampling_rate

def resample(waveform):
    transform = torchaudio.transforms.Resample(ORIG_SAMPLING_RATE, TGT_SAMPLING_RATE)
    waveform = transform(waveform)
    return waveform[0] The method loads a specified audio file and returns a multi-dimensional matrix of the audio data along with the sampling rate of the audio. All the audio files in the training data have a sampling rate of Hz. This "original" sampling rate is captured by the constant in . read_audio_data torch.Tensor 48000 ORIG_SAMPLING_RATE Step 2.4 The method is used to downsample audio data from a sampling rate of to the target sampling rate of . resample 48000 16000 Step 2.7 - CELL 7: Reading Test Data The seventh cell reads the test data index files for the recordings of male speakers and the recordings of female speakers using the method defined in . Set the seventh cell to: read_index_file_data Step 2.5 ### CELL 7: Read test data ###
test_data_male = read_index_file_data(TEST_DATA_PATH_MALE, "line_index.tsv")
test_data_female = read_index_file_data(TEST_DATA_PATH_FEMALE, "line_index.tsv") Step 2.8 - CELL 8: Generating Lists of Random Test Samples The eighth cell generates sets of random test samples using the method defined in . Set the eighth cell to: get_random_samples Step 2.5 ### CELL 8: Generate lists of random test samples ###
random_test_samples_male = get_random_samples(test_data_male, NUM_LOAD_FROM_EACH_SET)
random_test_samples_female = get_random_samples(test_data_female, NUM_LOAD_FROM_EACH_SET) Step 2.9 - CELL 9: Combining Test Data The ninth cell combines the male test samples and female test samples into a single list. Set the ninth cell to: ### CELL 9: Combine test data ###
all_test_samples = random_test_samples_male + random_test_samples_female Step 2.10 - CELL 10: Cleaning Transcription Test The tenth cell iterates over each test data sample and cleans the associated transcription text using the method defined in . Set the tenth cell to: clean_text Step 2.5 ### CELL 10: Clean text transcriptions ###
for index in range(len(all_test_samples)):
    all_test_samples[index][1] = clean_text(all_test_samples[index][1]) Step 2.11 - CELL 11: Loading Audio Data The eleventh cell loads each audio file specified in the list. Set the eleventh cell to: all_test_samples ### CELL 11: Load audio data ###
all_test_data = []

for index in range(len(all_test_samples)):
    speech_array, sampling_rate = read_audio_data(all_test_samples[index][0])
    all_test_data.append({
        "raw": speech_array,
        "sampling_rate": sampling_rate,
        "target_text": all_test_samples[index][1]
    }) Audio data is returned as a and stored in as a list of dictionaries. Each dictionary contains the audio data for a particular sample, the sampling rate, and the text transcription of the audio. torch.Tensor all_test_data Step 2.12 - CELL 12: Resampling Audio Data The twelfth cell resamples audio data to the target sampling rate of . Set the twelfth cell to: 16000 ### CELL 12: Resample audio data and cast to NumPy arrays ###
all_test_data = [{"raw": resample(sample["raw"]).numpy(), "sampling_rate": TGT_SAMPLING_RATE, "target_text": sample["target_text"]} for sample in all_test_data] Step 2.13 - CELL 13: Initializing Instance of Automatic Speech Recognition Pipeline The thirteenth cell initializes an instance of the HuggingFace library class. Set the thirteenth cell to: transformer pipeline ### CELL 13: Initialize instance of Automatic Speech Recognition Pipeline ###
transcriber = pipeline("automatic-speech-recognition", model = "YOUR_FINETUNED_MODEL_PATH") The parameter must be set to the path to finetuned model added to the Kaggle Notebook in , e.g.: model your Step 1.3 transcriber = pipeline("automatic-speech-recognition", model = "/kaggle/input/xls-r-300m-chilean-spanish/transformers/hardy-pine/1") Step 2.14 - CELL 14: Generating Predictions The fourteenth cell calls the initialized in the previous step on the test data to generate text predictions. Set the fourteenth cell to: transcriber ### CELL 14: Generate transcriptions ###
transcriptions = transcriber(all_test_data) Step 2.15 - CELL 15: Calculating WER Metrics The fifteenth cell calculates WER scores for each prediction as well as an overall WER score for all predictions. Set the fifteenth cell to: ### CELL 15: Calculate WER metrics ###
predictions = [transcription["text"] for transcription in transcriptions]
references = [transcription["target_text"][0] for transcription in transcriptions]

wers = []
for p in range(len(predictions)):
    wer = wer_metric.compute(predictions = [predictions[p]], references = [references[p]])
    wers.append(wer)

zipped = list(zip(predictions, references, wers))
df = pd.DataFrame(zipped, columns=["Prediction", "Reference", "WER"])
wer = wer_metric.compute(predictions = predictions, references = references) Step 2.16 - CELL 16: Printing WER Metrics The sixteenth and final cell simply prints the WER calculations from the previous step. Set the sixteenth cell to: ### CELL 16: Output WER metrics ###
pd.set_option("display.max_colwidth", None)
print(f"Overall WER: {wer}")
print(df) WER Analysis Since the notebook generates predictions on random samples of test data, the output will vary each time the notebook is run. The following output was generated on a run of the notebook with set to for a total of 6 test samples: NUM_LOAD_FROM_EACH_SET 3 Overall WER: 0.013888888888888888
                                                                              Prediction  \
0                                     quiero que me reserves el mejor asiento del teatro   
1                                         el llano en llamas es un clásico de juan rulfo   
2         el cuadro de los alcatraces es una de las pinturas más famosas de diego rivera   
3                           hay tres cafés que están abiertos hasta las once de la noche   
4  quiero que me recomiendes una dieta pero donde uno pueda comer algo no puras verduras   
5        cuántos albergues se abrieron después del terremoto del diecinueve de setiembre   

                                                                               Reference  \
0                                     quiero que me reserves el mejor asiento del teatro   
1                                         el llano en llamas es un clásico de juan rulfo   
2         el cuadro de los alcatraces es una de las pinturas más famosas de diego rivera   
3                           hay tres cafés que están abiertos hasta las once de la noche   
4  quiero que me recomiendes una dieta pero donde uno pueda comer algo no puras verduras   
5       cuántos albergues se abrieron después del terremoto del diecinueve de septiembre   

        WER  
0  0.000000  
1  0.000000  
2  0.000000  
3  0.000000  
4  0.000000  
5  0.090909 As can be seen, the model did an excellent job! It only made one error with the sixth sample (index ), mis-spelling the word as . Of course, running the notebook again with different test samples and, more importantly, a larger number of test samples, will produce different and more informative results. Nonetheless, this limited data suggests the model can perform well on different dialects of Spanish - i.e. it was trained on Chilean Spanish, but appears to perform well on Peruvian Spanish. 5 septiembre setiembre Conclusion If you are just learning how to work with wav2vec2 models, I hope that the guide and this guide were useful for you. As mentioned, the finetuned model generated by the Part 1 guide is not quite state-of-the-art, but should still prove useful for many applications. Happy building! Working With wav2vec2 Part 1 - Finetuning XLS-R for Automatic Speech Recognition

Working with wav2vec2 Part 2 - Running Inference on Finetuned ASR Models

wav2vec2 পার্ট 2-এর সাথে কাজ করা - Finetuned ASR মডেলগুলিতে অনুমান চালানো

Arbeiten mit wav2vec2 Teil 2 - Ausführen von Inferenzen auf fein abgestimmten ASR-Modellen

Trabajar con wav2vec2 Parte 2: Ejecución de inferencia en modelos ASR ajustados

Travailler avec wav2vec2 Partie 2 - Exécution d'inférence sur des modèles ASR affinés

wav2vec2 के साथ कार्य करना भाग 2 - फ़ाइनट्यून्ड ASR मॉडल पर इन्फ़रेंस चलाना

wav2vec2 の操作パート 2 - 微調整された ASR モデルでの推論の実行

wav2vec2 작업 2부 - 미세 조정된 ASR 모델에서 추론 실행

Trabalhando com wav2vec2 Parte 2 - Executando inferência em modelos ASR ajustados

Работа с wav2vec2. Часть 2. Выполнение вывода на точно настроенных моделях ASR

wav2vec2 ile Çalışmak Bölüm 2 - İnce Ayarlı ASR Modellerinde Çıkarım Çalıştırmak

Làm việc với wav2vec2 Phần 2 - Chạy suy luận trên các mô hình ASR đã được tinh chỉnh

使用 wav2vec2 第 2 部分 - 在经过微调的 ASR 模型上运行推理

The meme coin market is leading the current bull run, with the overall market cap recently surpassing $60 billion –  a two-year high. However, the premier tokens like Dogecoin and Shiba Inu were not the major catalysts of this latest rally in the meme coin market. Some of the newest projects launched just a few months ago, specifically Solana-based tokens, were among the highest gainers in the current market. This surging interest in meme coin trading will potentially pave the way for newer and more creative projects to enter the market. One such project is , a Solana-based meme token planning to send a dog to the moon. Laika Understanding Laika's Mission Laika is joining the latest meme coin developments, introducing a ‘community-driven reward ecosystem.’ The project presents a range of platforms and functions within this ecosystem, including a web3 wallet, a metaverse, and other classic tools. DeFi The token entered its presale phase in March with the , where each token purchase filled a helium balloon to launch a toy dog into the lower orbit. The project this mission last week, as over 30,000 viewers watched a toy dog fly into the earth’s lower Orbit. The campaign's next phase is ‘To the Moon,’ where Laika plans to send a dog to the International Space Station (ISS) and eventually to the lunar surface. Laika Moon Mission fulfilled This will be followed by the meme coin’s TGE (token generation event) in April and its potential listing on major exchanges. It’s a novel idea to show that meme coins can bring communities together and promote collective functions within the industry. Laika ignites a global movement, onboarding participants from a staggering 14+ countries in this campaign, most of whom are new to the crypto industry. The Goal: Education Through Engagement At its core, Laika's mission is educational. The project seeks to generate real excitement and discussions about the potential of community-driven launches and meme coins as tools for crypto education. By engaging participants in an interactive and fun way, Laika introduces a global audience, including the younger generation, to the basics of cryptocurrency. This approach challenges the status quo, offering a gamified learning experience that contrasts with the negative stereotypes often associated with meme coins and the broader crypto industry. A Community-Based Reward System The Laika project introduces a designed to incentivize participation and engagement within its platform. Here users can come together everyday and participate in various entertaining activities – whether it’s playing games in the metaverse or simply pressing a button to fill up a balloon with helium. In February, they also launched an AR challenge on X under . community-driven reward ecosystem #LaikaWorldChallenge At the same time, the project provides general DeFi functions such as NFT staking. Users can stake their assets to receive ‘boosts,’ likely enhancing their benefits or standing within the community. Laika will also hold regular pool raffles, where participants can win popular meme coins such as DOGE, SHIBA, and BABYDOGE. Users can also stake the native token to earn an APY of up to 20%, providing a financial incentive for long-term investment. Alongside these, a point system will provide daily claims, engagement, and in-game feats, likely allowing users to accumulate points for various actions within the platform. The ruffDAO will manage the project to support community-driven decision-making and growth opportunities. There’s also an ongoing , where the Laika team will reward the funniest community-voted memes on X (formerly Twitter). It shows the project's commitment to the classic social media meme culture that influenced the entire meme coin phenomenon. meme contest Key Features of the Laika Project At the heart of Laika's ecosystem is the Laika Foundation, dedicated to creating the biggest community. The foundation supports large-scale projects accessible to everyone within and outside the Web3 space. Laika wants to demonstrate that crypto adoption is not complex, rather, it can happen gradually with continuous learning, engagement, and fun. The Laika SuperVerse extends this mission, offering fully immersive experiences for exploration, learning, and interaction, bridging the Web2 and Web3 worlds. Laika introduces a unique token distribution model that maximizes community participation and ownership. Coupled with cutting-edge 2D, 3D, and AR NFTs, Laika offers a new dimension of digital asset interaction backed by DeFi functionality. There’s also the Laika Wallet and gaming studio, which will facilitate Web3 transactions and introduce new Web3 mobile games. The is already available for download on the Google Play Store, soon it will be listed in the App Store. Laika Wallet Global Brand and Charity Programs Beyond its digital ventures, Laika aims to expand into the physical world through key brand partnerships, merchandise, and events. The project also prioritizes social impact, with community-driven initiatives to build dog shelters globally, highlighting its commitment to positive change both online and offline. There is a significant charity aspect to the project, as the team plans to hold an action for its toy dog that traveled to the lower orbit last week. All proceeds from the auction will go towards supporting a dog shelter. Bottom Line As the Laika project takes its first steps towards its moon mission, it represents a bold experiment in the power of community, education, and innovation within crypto space. While a large number of meme coin projects tend to thrive on speculative hype and liquidity manipulation, Laika has shown true commitment to its roadmap and campaign. The rapidly growing meme coin market has great scope for the latest entries like Laika. It will be an intriguing project for the meme crypto community, one that investors will surely keep a close eye on.

Laika Goes to the Moon: A Novel Approach to Memecoin Education and Community Engagement

লাইকা চাঁদে যায়: মেমেকয়েন শিক্ষা এবং সম্প্রদায়ের ব্যস্ততার জন্য একটি অভিনব পদ্ধতি

Laika fliegt zum Mond: Ein neuartiger Ansatz zur Aufklärung über Memecoins und zur Einbindung der Community

Laika va a la luna: un enfoque novedoso para la educación y la participación comunitaria de Memecoin

Laika va sur la Lune : une nouvelle approche de l'éducation Memecoin et de l'engagement communautaire

लाइका चाँद पर जाती है: मेमेकोइन शिक्षा और सामुदायिक सहभागिता के लिए एक नया दृष्टिकोण

ライカが月へ行く: ミームコイン教育とコミュニティ参加への斬新なアプローチ

Laika는 달에 간다: Memecoin 교육 및 커뮤니티 참여에 대한 새로운 접근 방식

Laika vai à Lua: uma nova abordagem para a educação do Memecoin e o envolvimento da comunidade

Laika отправляется на Луну: новый подход к образованию Memecoin и взаимодействию с сообществом

Laika Aya Gidiyor: Memecoin Eğitimi ve Topluluk Katılımına Yeni Bir Yaklaşım

Laika lên mặt trăng: Một cách tiếp cận mới đối với giáo dục Memecoin và sự gắn kết với cộng đồng

莱卡登上月球：Memecoin 教育和社区参与的新方法

Navigating the Nuances of Median Statistics in Sliding Windows Following the whirlwind success of our into the algorithmic enigmas of coding challenges, it's time to delve into another intriguing puzzle that has garnered attention in coding interviews and beyond. Today, we're shifting our focus from the tangible geometry of cube surfaces to the abstract realms of statistics, specifically, the calculation of medians within a sliding window. But what exactly is a median, and why does it hold more significance than the average in certain scenarios? previous exploration A median represents the middle value in a dataset when arranged in ascending order. Unlike the mean, which sums all values and divides by their count, the median is less susceptible to outliers—a property that renders it invaluable in datasets with high variance and a low signal-to-noise ratio, such as those found in financial markets. The formula for the median in a sorted list of numbers depends on the count being odd or even: for an odd count, it's the middle number; for an even count, it's the average of the two middle numbers. Let's be real: when it comes to slicing and dicing data over time, window-based stats are pretty much the superhero of the story. , understanding the dynamic median of a dataset over a specified window offers insights that static analysis could miss. Whether in high-frequency trading (HFT), machine learning, time series analysis, or monitoring systems The Problem: Median in a Sliding Window Consider a sliding window moving across a series of numerical data points. Within this window, we aim to calculate the median of the data points at each step, providing a continuous view of the dataset's central tendency as the window progresses. This task is not only a staple of coding interviews but also finds practical application in fields as diverse as HFT, machine learning, and real-time data analysis. Example: Given the input series , with a window size of , the medians are [ -7, -7, -6, 1, -6, -5, -10, 4, 6, 8] 3 [-7.0, -6.0, -6.0, -5.0, -6.0, -5.0, 4.0, 6.0] This task's relevance extends far beyond the confines of coding challenges, impacting various domains such as HFT, where accurate and timely data analysis can yield significant financial outcomes, machine learning models that depend on robust feature extraction from noisy datasets, and the critical evaluation of trends in time series data, vital for forecasting and anomaly detection. The application of such an algorithm in real-world scenarios underscores the overarching importance of median statistics in sliding windows, offering a more reliable measure of central tendency amidst data fluctuations. As we proceed, this article will unravel the intricacies of calculating medians in sliding windows, a testament to the ongoing need for adept problem-solving and algorithmic prowess in the ever-evolving field of software development and data analysis. Stay tuned as we dissect the problem, outline a solution, and delve into the practical applications of this fascinating statistical challenge. Essential Data Structures: Binary Heaps, Queues, and Lazy Deletion Dictionaries To address the median in a sliding window challenge, we deploy an arsenal of sophisticated data structures, each serving a unique role in the algorithm's design and functionality. Here's a closer look at these components: : At the core of our solution are two types of binary heaps: the min-heap and the max-heap. A binary heap is a complete binary tree where each node is smaller (min-heap) or larger (max-heap) than its children, ensuring the smallest or largest element is always at the root. Picture a binary heap as a family tree where each parent is either the runt (in a min-heap) or the giant (in a max-heap) compared to their kids. This property is key to efficiently managing the data within the sliding window, allowing quick access to the median. The max-heap is used to store the lower half of the numbers in the window, ensuring quick access to the smaller values, while the min-heap holds the upper half, managing the larger values. This division enables us to efficiently locate the median value by looking at the roots of these heaps. Binary Heaps : This data structure operates on a first-in, first-out (FIFO) principle, making it perfect for tracking the order of elements as they enter and exit the sliding window. As new data points arrive, they're enqueued (added to the rear), and as the window moves, the oldest data points are dequeued (removed from the front). This mechanism ensures we always know which element to remove from our binary heaps when it exits the window. Queue : Lazy deletion is an optimization technique used to handle the removal of elements from our heaps. Instead of immediately removing an element from a heap when it exits the window, we mark it for deletion in a dictionary. This approach allows us to defer the actual removal until necessary (i.e., when the element reaches the root of the heap), reducing the overhead associated with heap operations. The dictionary tracks the count of how many times each element is marked for deletion, enabling efficient management of duplicates and ensuring that our heaps always reflect the current state of the sliding window. Dictionary for Lazy Deletions By integrating these data structures, we construct a robust framework capable of dynamically calculating medians in a sliding window, balancing efficiency with accuracy. This strategic assembly not only facilitates the median calculation but also exemplifies the power of data structure synergy in solving complex algorithmic problems. The crux of the median calculation lies in maintaining a balanced view of the data as it changes dynamically. By keeping half of the elements in a max-heap and the other half in a min-heap, we can efficiently calculate the median by examining the tops of these heaps. This approach adapts well to sliding window scenarios, where data points continuously enter and exit the window, necessitating real-time updates to the median calculation. Algorithmic Approach: Building a Composite Data Structure Our solution architecture combines the aforementioned data structures to form a composite structure capable of efficiently solving our median calculation problem. This setup involves: : When a new element arrives, we determine which heap to add it to based on its value relative to the current median. This step might ruffle some feathers or, well, disturb the heap harmony for a sec. Adding Elements : To ensure the heaps remain balanced (i.e., have the same number of elements or differ by at most one), we may need to move elements between them after adding a new element or removing an old one. Rebalancing Heaps : As the window slides, the oldest element (the one that exits the window) must be removed. We use the lazy deletion strategy here, marking elements for deletion in our dictionary and physically removing them from the heaps only when they reach the top, thereby avoiding unnecessary reheapifications. Removing Old Elements : This step involves removing the elements marked for deletion from the heaps, ensuring that our median calculation remains accurate and our data structure does not grow unnecessarily large. Lazy Cleaning Working with the Composite Structure Implementing this composite structure involves careful management of each component: : Setting up the min-heap, max-heap, queue, and lazy deletion dictionary. Initialization An important aspect of our approach that readers should note is the use of a simple yet effective trick for converting between max heaps and min heaps. We achieve this by applying a unary minus to the elements before adding them to the heap. This allows us to use Python's standard min-heap provided by as a max heap, by adding and removing elements with their sign inverted. This approach significantly simplifies managing the two heaps to maintain the median in a sliding window, facilitating the process of adding new elements, balancing the heaps, and adjusting the median as elements are added or removed. heapq : Populating our data structure with an initial set of elements to start the median calculation process. Warm-up : Efficiently determining the median based on the current state of the min-heap and max-heap. Calculating the Median : Adjusting the heaps to maintain balance after adding or removing elements. Rebalancing : Adding new elements to the appropriate heap based on their value relative to the median, and removing elements from the queue (and marking them for lazy deletion) as they exit the window. Element Addition and Removal : Periodically executing lazy deletion to remove marked elements from the heaps, ensuring the efficiency and accuracy of the median calculation. Lazy Cleanup Through this comprehensive approach, we address the challenges of calculating the median in a dynamic, sliding window environment, showcasing the power of combining simple data structures to solve complex algorithmic problems. As we continue, we will delve deeper into the specifics of each component within the class, providing a detailed guide to its implementation and usage. MedianWindow The Code: MedianWindow Class Implementation Having explored the theoretical underpinnings and the essential data structures that make our solution viable, it's time to present the implementation of the class. This class is the cornerstone of our approach, embodying the algorithmic strategy we've outlined to dynamically calculate medians within a sliding window of data points. MedianWindow import heapq

class MedianWindow:
    def __init__(self, win_size):
        self.win_size = win_size
        self.min_heap = []
        self.max_heap = []
        self.queue = deque([])
        self.heap_dict = defaultdict(int)
        self.balance = 0

    def warmup(self, batch):
        for x in batch:
            self.queue.append(x)
            heapq.heappush(
                self.max_heap, 
                -x
            )
            heapq.heappush(
                self.min_heap, 
                -heapq.heappop(self.max_heap)
            )
            if len(self.min_heap) > len(self.max_heap):
                heapq.heappush(
                    self.max_heap, 
                    -heapq.heappop(self.min_heap)
                )

    @property
    def median(self):
        if self.win_size % 2 == 1:
            return -self.max_heap[0]
        else:
            return (-self.max_heap[0] + self.min_heap[0]) / 2

    def rebalance(self):
        if self.balance < 0:
            heapq.heappush(
                self.max_heap, 
                -heapq.heappop(self.min_heap)
            )
        elif self.balance > 0:
            heapq.heappush(
                self.min_heap, 
                -heapq.heappop(self.max_heap)
            )

    def push(self, x):
        self.queue.append(x)
        oldest = self.queue[0]
        self.balance = -1 if oldest <= self.median else 1
        if x <= self.median:
            self.balance += 1
            heapq.heappush(
                self.max_heap, 
                -x
            )
        else:
            self.balance -= 1
            heapq.heappush(
                self.min_heap, 
                x
            )
        self.rebalance()

    def should_remove_max_heap_top(self):
        h = self.max_heap
        return h and self.heap_dict[-self.max_heap[0]] > 0

    def should_remove_min_heap_top(self):
        h = self.min_heap
        return h and self.heap_dict[self.min_heap[0]] > 0

    def lazy_clean(self):
        while self.should_remove_max_heap_top():
            self.heap_dict[-self.max_heap[0]] -= 1
            heapq.heappop(self.max_heap)

        while self.should_remove_min_heap_top():
            self.heap_dict[self.min_heap[0]] -= 1
            heapq.heappop(self.min_heap)

    def pop(self):
        prev_num = self.queue.popleft()
        self.heap_dict[prev_num] += 1
        self.lazy_clean()
        return prev_num Example of usage: w = MedianWindow(k)
w.warmup(data[:k])
result = [w.median]
for i in range(k, len(data)):
    w.push(data[i])
    old_elem = w.pop()
    result.append(w.median) This implementation encapsulates the complexity of managing a sliding window of data points and efficiently calculating the median. The class utilizes binary heaps to maintain a running median, a queue to track the sliding window, and a dictionary for lazy deletions, illustrating a practical application of these data structures in solving a non-trivial algorithmic challenge. Analyzing Time Complexity Understanding the time complexity of our solution is crucial for assessing its efficiency and suitability for real-world applications. The MedianWindow class's operations can be analyzed as follows: ): The constructor has a constant time complexity, O(1), as it merely sets up the initial state without processing any data. Initialization ( __init__ ): This method has a time complexity of O(k log k), where k is the size of the sliding window. Each insertion into a heap is O(log k), and we perform k such insertions. Warm-up ( warmup ): Retrieving the median has a constant time complexity, O(1), since it involves returning the top element from one or both heaps, operations that do not depend on the size of the window. Median Calculation ( median ): The rebalance operation, invoked after adding or removing elements, maintains the heaps in a balanced state. This operation has a logarithmic time complexity, O(log k), due to the heap insertions and deletions. Rebalancing ( rebalance ): Adding a new element involves inserting it into one of the heaps and possibly rebalancing. The overall time complexity is O(log k) due to these heap operations. Adding Elements ( push ): The removal process has a logarithmic time complexity, O(log k), for similar reasons to the addition process. However, because of lazy deletion, the actual removal from the heap is deferred, potentially reducing the frequency of heap reorganizations. Removing Elements ( pop ): This operation's time complexity can vary. In the worst case, if many elements are marked for deletion, cleaning them out when they reach the top of the heap can take O(log k) per element. However, this cost is amortized over the operations that caused the elements to be marked, mitigating the impact on performance. Lazy Cleanup ( lazy_clean The efficiency of the MedianWindow class hinges on the logarithmic time complexity of heap operations, making it well-suited for handling sliding windows of data where the window size is significantly smaller than the total number of data points. This property is particularly beneficial in applications like high-frequency trading and real-time data analysis, where performance and timely data processing are paramount. In the next section, we'll delve into the practical challenges one might encounter when implementing this algorithm and suggest potential optimizations and real-world applications, underscoring the practical relevance of mastering data structures and algorithms through the lens of the sliding window median problem. Factors Affecting Performance The efficiency of the MedianWindow algorithm hinges on several critical factors: : The core of our performance lies in the heap operations, primarily insertions and deletions, each with a logarithmic time complexity of O(log k), where k is the window size. Efficient management of these operations is key to maintaining the algorithm's overall performance. Heap Operations : Our approach to lazy deletion significantly impacts performance by deferring the removal of elements from the heaps until absolutely necessary. While this strategy reduces immediate computational overhead, its effectiveness is contingent on the frequency and distribution of elements marked for deletion. Lazy Deletion Strategy : The size of the sliding window and the distribution of input data can affect performance. Larger windows increase the complexity of heap operations, and skewed data distributions may lead to unbalanced heaps, necessitating more frequent rebalancing operations. Window Size and Data Distribution Practical Challenges To solidify understanding and encourage hands-on experimentation, here are some practical challenges for readers: : Enrich the algorithm's capabilities by enabling the calculation of any percentile, not just the median. This enhancement involves introducing a new parameter to adjust the algorithm accordingly. To tackle this challenge, consider how the current structure, based on min and max heaps, could be generalized or modified to provide efficient access to arbitrary percentiles. This extension not only broadens the algorithm's applicability but also challenges the reader to think about data structures in a more flexible and adaptable way. Percentile Calculation : Modify the MedianWindow class to improve memory efficiency when dealing with large datasets. Explore strategies to minimize the memory footprint of heap structures. Memory Efficiency : Extend the algorithm to support median calculations on datasets with non-numeric types, such as timestamps or custom objects, ensuring type consistency and appropriate comparison mechanisms. Diverse Data Types Algorithmic Improvements and Optimization While the MedianWindow algorithm demonstrates robust capabilities, there's always room for optimization: : Implement a heuristic approach to rebalancing the heaps based on anticipated data patterns or historical distributions. This could reduce the number of rebalancing operations needed, enhancing performance. Heuristic Rebalancing : For datasets where real-time processing is not critical, modify the algorithm to process data in batches. This can significantly reduce the computational overhead associated with heap operations. Batch Processing : Explore the feasibility of parallelizing heap operations or the processing of incoming data points. While the inherently sequential nature of median calculation may limit parallelism, certain preprocessing tasks or the lazy cleanup operation might be parallelized to improve throughput. Parallel Processing : Investigate the use of alternative data structures, such as Fibonacci heaps or soft heaps, which might offer better amortized time complexities for certain operations, albeit at the cost of increased implementation complexity or specific trade-offs. Alternative Data Structures In concluding our exploration of the Dynamic Median Calculation within the Sliding Windows algorithm, we've not just tackled a complex problem but have also highlighted the critical thinking skills vital in modern data analysis. This endeavor goes beyond mere computation; it is an exercise in enhancing our algorithmic agility and intellectual flexibility. Stay engaged as we continue to explore the vast frontiers of algorithmic development, each step bringing us closer to mastering the art of problem-solving in an ever-evolving digital world.

Coding Interview Prep: Mastering Median Statistics in Sliding Windows

581 Stories To Learn About Non Fiction

A Treatise on Painting by da Vinci Leonardo, is part of the HackerNoon Books Series. You can jump to any chapter in this book . LANDSCAPE. here MISCELLANEOUS OBSERVATIONS. LANDSCAPE. Chap. CCCXXIII.— Of Objects seen at a Distance. Any dark object will appear lighter when removed to some distance from the eye. It follows, by the contrary reason, that a dark object will appear still darker when brought nearer to the eye. Therefore the inferior parts of any object whatever, placed in thick air, will appear farther from the eye at the bottom than at the top; for that reason the lower parts of a mountain appear farther off than its top, which is in reality the farthest. Chap. CCCXXIV.— Of a Town seen through a thick Air. The eye which, looking downwards, sees a town immersed in very thick air, will perceive the top of the buildings darker, but more distinct than the bottom. The tops detach against a light ground, because they are seen against the low and thick air which is beyond them. This is a consequence of what has been explained in the preceding chapter. Chap. CCCXXV.— How to draw a Landscape. Contrive that the trees in your landscape be half in shadow and half in the light. It is better to represent them as when the sun is veiled with thin clouds, because in that case the trees receive a general light from the sky, and are darkest in those parts which are nearest to the earth. Chap. CCCXXVI.— Of the Green of the Country. Of the greens seen in the country, that of trees and other plants will appear darker than that of fields and meadows, though they may happen to be of the same quality. Chap. CCCXXVII.— What Greens will appear most of a blueish Cast. Those greens will appear to approach nearest to blue which are of the darkest shade when remote. This is proved by the seventh proposition, which says, that blue is composed of black and white seen at a great distance. Chap. CCCXXVIII.— The Colour of the Sea from different Aspects. When the sea is a little ruffled it has no sameness of colour; for whoever looks at it from the shore, will see it of a dark colour, in a greater degree as it approaches towards the horizon, and will perceive also certain lights moving slowly on the surface like a flock of sheep. Whoever looks at the sea from on board a ship, at a distance from the land, sees it blue. Near the shore it appears darkish, on account of the colour of the earth reflected by the water, as in a looking-glass; but at sea the azure of the air is reflected to the eye by the waves in the same manner. Chap. CCCXXIX.— Why the same Prospect appears larger at some Times than at others. Objects in the country appear sometimes larger and sometimes smaller than they actually are, from the circumstance of the air interposed between the eye and the horizon, happening to be either thicker or thinner than usual. Of two horizons equally distant from the eye, that which is seen through the thicker air will appear farther removed; and the other will seem nearer, being seen through a thinner air. Objects of unequal size, but equally distant, will appear equal if the air which is between them and the eye be of proportionable inequality of thickness, viz. if the thickest air be interposed between the eye and the smallest of the objects. This is proved by the perspective of colours, which is so deceitful that a mountain which would appear small by the compasses, will seem larger than a small hill near the eye; as a finger placed near the eye will cover a large mountain far off. Chap. CCCXXX.— Of Smoke. Smoke is more transparent, though darker towards the extremities of its waves than in the middle. It moves in a more oblique direction in proportion to the force of the wind which impels it. Different kinds of smoke vary in colour, as the causes that produce them are various. Smoke never produces determined shadows, and the extremities are lost as they recede from their primary cause. Objects behind it are less apparent in proportion to the thickness of the smoke. It is whiter nearer its origin, and bluer towards its termination. Fire appears darker, the more smoke there is interposed between it and the eye. Where smoke is farther distant, the objects are less confused by it. It encumbers and dims all the landscape like a fog. Smoke is seen to issue from different places, with flames at the origin, and the most dense part of it. The tops of mountains will be more seen than the lower parts, as in a fog. Chap. CCCXXXI.— In what Part Smoke is lightest. Smoke which is seen between the sun and the eye will be lighter and more transparent than any other in the landscape. The same is observed of dust, and of fog; while, if you place yourself between the sun and those objects, they will appear dark. Chap. CCCXXXII.— Of the Sun-beams passing through the Openings of Clouds. The sun-beams which penetrate the openings interposed between clouds of various density and form, illuminate all the places over which they pass, and tinge with their own colour all the dark places that are behind: which dark places are only seen in the intervals between the rays. Chap. CCCXXXIII.— Of the Beginning of Rain. When the rain begins to fall, it tarnishes and darkens the air, giving it a dull colour, but receives still on one side a faint light from the sun, and is shaded on the other side, as we observe in clouds; till at last it darkens also the earth, depriving it entirely of the light of the sun. Objects seen through the rain appear confused and of undetermined shape, but those which are near will be more distinct. It is observable, that on the side where the rain is shaded, objects will be more clearly distinguished than where it receives the light; because on the shady side they lose only their principal lights, whilst on the other they lose both their lights and shadows, the lights mixing with the light part of the rain, and the shadows are also considerably weakened by it. Chap. CCCXXXIV.— The Seasons are to be observed. In Autumn you will represent the objects according as it is more or less advanced. At the beginning of it the leaves of the oldest branches only begin to fade, more or less, however, according as the plant is situated in a fertile or barren country; and do not imitate those who represent trees of every kind (though at equal distance) with the same quality of green. Endeavour to vary the colour of meadows, stones, trunks of trees, and all other objects, as much as possible, for Nature abounds in variety . ad infinitum Chap. CCCXXXV.— The Difference of Climates to be observed. Near the sea-shore, and in southern parts, you will be careful not to represent the Winter season by the appearance of trees and fields, as you would do in places more inland, and in northern countries, except when these are covered with ever-greens, which shoot afresh all the year round. Chap. CCCXXXVI.— Of Dust. Dust becomes lighter the higher it rises, and appears darker the less it is raised, when it is seen between the eye and the sun. Chap. CCCXXXVII.— How to represent the Wind. In representing the effect of the wind, besides the bending of trees, and leaves twisting the wrong side upwards, you will also express the small dust whirling upwards till it mixes in a confused manner with the air. Chap. CCCXXXVIII.— Of a Wilderness. Those trees and shrubs which are by their nature more loaded with small branches, ought to be touched smartly in the shadows, but those which have larger foliage, will cause broader shadows. Chap. CCCXXXIX.— Of the Horizon seen in the Water. By the sixth proposition, the horizon will be seen in the water as in a looking-glass, on that side which is opposite the eye. And if the painter has to represent a spot covered with water, let him remember that the colour of it cannot be either lighter or darker than that of the neighbouring objects. Chap. CCCXL.— Of the Shadow of Bridges on the Surface of the Water. The shadows of bridges can never be seen on the surface of the water, unless it should have lost its transparent and reflecting quality, and become troubled and muddy; because clear water being polished and smooth on its surface, the image of the bridge is formed in it as in a looking-glass, and reflected in all the points situated between the eye and the bridge at equal angles; and even the air is seen under the arches. These circumstances cannot happen when the water is muddy, because it does not reflect the objects any longer, but receives the shadow of the bridge in the same manner as a dusty road would receive it. Chap. CCCXLI.— How a Painter ought to put in Practice the Perspective of Colours. To put in practice that perspective which teaches the alteration, the lessening, and even the entire loss of the very essence of colours, you must take some points in the country at the distance of about sixty-five yards from each other; as trees, men, or some other remarkable objects. In regard to the first tree, you will take a glass, and having fixed that well, and also your eye, draw upon it, with the greatest accuracy, the tree you see through it; then put it a little on one side, and compare it closely with the natural one, and colour it, so that in shape and colour it may resemble the original, and that by shutting one eye they may both appear painted, and at the same distance. The same rule may be applied to the second and third tree at the distance you have fixed. These studies will be very useful if managed with judgment, where they may be wanted in the offscape of a picture. I have observed that the second tree is less by four fifths than the first, at the distance of thirteen yards. Chap. CCCXLII.— Various Precepts in Painting. The superficies of any opake body participates of the colour of the transparent medium interposed between the eye and such body, in a greater or less degree, in proportion to the density of such medium and the space it occupies. The outlines of opake bodies will be less apparent in proportion as those bodies are farther distant from the eye. That part of the opake body will be the most shaded, or lightest, which is nearest to the body that shades it, or gives it light. The surface of any opake body participates more or less of the colour of that body which gives it light, in proportion as the latter is more or less remote, or more or less strong. Objects seen between lights and shadows will appear to have greater relievo than those which are placed wholly in the light, or wholly in shadow. When you give strength and precision to objects seen at a great distance, they will appear as if they were very near. Endeavour that your imitation be such as to give a just idea of distances. If the object in nature appear confused in the outlines, let the same be observed in your picture. The outlines of distant objects appear undetermined and confused, for two reasons: the first is, that they come to the eye by so small an angle, and are therefore so much diminished, that they strike the sight no more than small objects do, which though near can hardly be distinguished, such as the nails of the fingers, insects, and other similar things: the second is, that between the eye and the distant objects there is so much air interposed, that it becomes thick; and, like a veil, tinges the shadows with its own whiteness, and turns them from a dark colour to another between black and white, such as azure. Although, by reason of the great distance, the appearance of many things is lost, yet those things which receive the light from the sun will be more discernible, while the rest remain enveloped in confused shadows. And because the air is thicker near the ground, the things which are lower will appear confused; and . vice versâ When the sun tinges the clouds on the horizon with red, those objects which, on account of their distance, appear blueish, will participate of that redness, and will produce a mixture between the azure and red, which renders the prospect lively and pleasant; all the opake bodies which receive that light will appear distinct, and of a reddish colour, and the air, being transparent, will be impregnated with it, and appear of the colour of lilies. The air which is between the earth and the sun when it rises or sets, will always dim the objects it surrounds, more than the air any where else, because it is whiter. It is not necessary to mark strongly the outlines of any object which is placed upon another. It ought to detach of itself. If the outline or extremity of a white and curved surface terminate upon another white body, it will have a shade at that extremity, darker than any part of the light; but if against a dark object, such outline, or extremity, will be lighter than any part of the light. Those objects which are most different in colour, will appear the most detached from each other. Those parts of objects which first disappear in the distance, are extremities similar in colour, and ending one upon the other, as the extremities of an oak tree upon another oak similar to it. The next to disappear at a greater distance are, objects of mixed colours, when they terminate one upon the other, as trees, ploughed fields, walls, heaps of rubbish, or of stones. The last extremities of bodies that vanish are those which, being light, terminate upon a dark ground; or being dark, upon a light ground. Of objects situated above the eye, at equal heights, the farthest removed from the eye will appear the lowest; and if situated below the eye, the nearest to it will appear the lowest. The parallel lines situated sidewise will concur to one point. Those objects which are near a river, or a lake, in the distant part of a landscape, are less apparent and distinct than those that are remote from them. Of bodies of equal density, those that are nearest to the eye will appear thinnest, and the most remote thickest. A large eye-ball will see objects larger than a small one. The experiment may be made by looking at any of the celestial bodies, through a pin-hole, which being capable of admitting but a portion of its light, it seems to diminish and lose of its size in the same proportion as the pin-hole is smaller than the usual apparent size of the object. A thick air interposed between the eye and any object, will render the outlines of such object undetermined and confused, and make it appear of a larger size than it is in reality; because the linear perspective does not diminish the angle which conveys the object to the eye. The aerial perspective carries it farther off, so that the one removes it from the eye, while the other preserves its magnitude. When the sun is in the West the vapours of the earth fall down again and thicken the air, so that objects not enlightened by the sun remain dark and confused, but those which receive its light will be tinged yellow and red, according to the sun’s appearance on the horizon. Again, those that receive its light are very distinct, particularly public buildings and houses in towns and villages, because their shadows are dark, and it seems as if those parts which are plainly seen were coming out of confused and undetermined foundations, because at that time every thing is of one and the same colour, except what is enlightened by the sun. Any object receiving the light from the sun, receives also the general light; so that two kinds of shadows are produced: the darkest of the two is that which happens to have its central line directed towards the centre of the sun. The central lines of the primitive and secondary lights are the same as the central lines of the primitive and secondary shadows. The setting sun is a beautiful and magnificent object when it tinges with its colour all the great buildings of towns, villages, and the top of high trees in the country. All below is confused and almost lost in a tender and general mass; for, being only enlightened by the air, the difference between the shadows and the lights is small, and for that reason it is not much detached. But those that are high are touched by the rays of the sun, and, as was said before, are tinged with its colour; the painter therefore ought to take the same colour with which he has painted the sun, and employ it in all those parts of his work which receive its light. It also happens very often, that a cloud will appear dark without receiving any shadow from a separate cloud, according to the situation of the eye; because it will see only the shady part of the one, while it sees both the enlightened and shady parts of the other. Of two objects at equal height, that which is the farthest off will appear the lowest. Observe the first cloud in the cut, though it is lower than the second, it appears as if it were higher. This is demonstrated by the section of the pyramidical rays of the low cloud at M A, and the second (which is higher) at N M, below M A. This happens also when, on account of the rays of the setting or rising sun, a dark cloud appears higher than another which is light. Chap. CCCXLIII.— The Brilliancy of a Landscape. The vivacity and brightness of colours in a landscape will never bear any comparison with a landscape in nature when illumined by the sun, unless the picture be placed so as to receive the same light from the sun itself. About HackerNoon Book Series: We bring you the most important technical, scientific, and insightful public domain books. This book is part of the public domain. da Vinci Leonardo (2014). A Treatise on Painting. Urbana, Illinois: Project Gutenberg. Retrieved https://www.gutenberg.org/cache/epub/46915/pg46915-images.html This eBook is for the use of anyone anywhere at no cost and with almost no restrictions whatsoever. You may copy it, give it away or re-use it under the terms of the Project Gutenberg License included with this eBook or online at , located at . www.gutenberg.org https://www.gutenberg.org/policy/license.html

Of Objects seen at a Distance

BRIDAL VISITS AND ADIEUX.

A Treatise on Painting by da Vinci Leonardo, is part of the HackerNoon Books Series. You can jump to any chapter in this book . EXPRESSION and CHARACTER. here EXPRESSION and CHARACTER. Chap. CLXV.— Of expressive Motions. Let your figures have actions appropriated to what they are intended to think or say, and these will be well learnt by imitating the deaf, who by the motion of their hands, eyes, eyebrows, and the whole body, endeavour to express the sentiments of their mind. Do not ridicule the thought of a master without a tongue teaching you an art he does not understand; he will do it better by his expressive motions, than all the rest by their words and examples. Let then the painter, of whatever school, attend well to this maxim, and apply it to the different qualities of the figures he represents, and to the nature of the subject in which they are actors. Chap. CLXVI.— How to paint Children. Children are to be represented with quick and contorted motions, when they are sitting; but when standing, with fearful and timid motions. Chap. CLXVII.— How to represent old Men. Old men must have slow and heavy motions; their legs and knees must be bent when they are standing, and their feet placed parallel and wide asunder. Let them be bowed downwards, the head leaning much forward, and their arms very little extended. Chap. CLXVIII.— How to paint old Women. Old women, on the contrary, are to be represented bold and quick, with passionate motions, like furies. But the motions are to appear a great deal quicker in their arms than in their legs. Chap. CLXIX.— How to paint Women. Women are to be represented in modest and reserved attitudes, with their knees rather close, their arms drawing near each other, or folded about the body; their heads looking downwards, and leaning a little on one side. Chap. CLXX.— Of the Variety of Faces. The countenances of your figures should be expressive of their different situations: men at work, at rest, weeping, laughing, crying out, in fear, or joy, and the like. The attitudes also, and all the members, ought to correspond with the sentiment expressed in the faces. Chap.CLXXI.— The Parts of the Face, and their Motions. The motions of the different parts of the face, occasioned by sudden agitations of the mind, are many. The principal of these are, Laughter, Weeping, Calling out, Singing, either in a high or low pitch, Admiration, Anger, Joy, Sadness, Fear, Pain, and others, of which I propose to treat. First, of Laughing and Weeping, which are very similar in the motion of the mouth, the cheeks, the shutting of the eyebrows, and the space between them; as we shall explain in its place, in treating of the changes which happen in the face, hands, fingers, and all the other parts of the body, as they are affected by the different emotions of the soul; the knowledge of which is absolutely necessary to a painter, or else his figures may be said to be twice dead. But it is very necessary also that he be careful not to fall into the contrary extreme; giving extraordinary motions to his figures, so that in a quiet and peaceable subject, he does not seem to represent a battle, or the revellings of drunken men: but, above all, the actors in any point of history must be attentive to what they are about, or to what is going forward; with actions that denote admiration, respect, pain, suspicion, fear, and joy, according as the occasion, for which they are brought together, may require. Endeavour that different points of history be not placed one above the other on the same canvass, nor walls with different horizons, as if it were a jeweller’s shop, shewing the goods in different square caskets. Chap. CLXXII.— Laughing and Weeping. Between the expression of laughter and that of weeping there is no difference in the motion of the features either in the eyes, mouth, or cheeks; only in the ruffling of the brows, which is added when weeping, but more elevated and extended in laughing. One may represent the figure weeping as tearing his clothes, or some other expression, as various as the cause of his feeling may be; because some weep for anger, some through fear, others for tenderness and joy, or for suspicion; some for real pain and torment; whilst others weep through compassion, or regret at the loss of some friend and near relation. These different feelings will be expressed by some with marks of despair, by others with moderation; some only shed tears, others cry aloud, while another has his face turned towards heaven, with his hand depressed, and his fingers twisted. Some again will be full of apprehension, with their shoulders raised up to their ears, and so on, according to the above causes. Those who weep, raise the brows, and bring them close together above the nose, forming many wrinkles on the forehead, and the corners of the mouth are turned downwards. Those who laugh have them turned upwards, and the brows open and extended. Chap. CLXXIII.— Of Anger. If you represent a man in a violent fit of anger, make him seize another by the hair, holding his head writhed down against the ground, with his knee fixed upon the ribs of his antagonist; his right arm up, and his fist ready to strike; his hair standing on end, his eyebrows low and straight; his teeth close, and seen at the corner of the mouth; his neck swelled, and his body covered in the Abdomen with creases, occasioned by his bending over his enemy, and the excess of his passion. Chap. CLXXIV.— Despair. The last act of despondency is, when a man is in the act of putting a period to his own existence. He should be represented with a knife in one hand, with which he has already inflicted the wound, and tearing it open with the other. His garments and hair should be already torn. He will be standing with his feet asunder, his knees a little bent, and his body leaning forward, as if ready to fall to the ground. About HackerNoon Book Series: We bring you the most important technical, scientific, and insightful public domain books. This book is part of the public domain. da Vinci Leonardo (2014). A Treatise on Painting. Urbana, Illinois: Project Gutenberg. Retrieved https://www.gutenberg.org/cache/epub/46915/pg46915-images.html This eBook is for the use of anyone anywhere at no cost and with almost no restrictions whatsoever. You may copy it, give it away or re-use it under the terms of the Project Gutenberg License included with this eBook or online at , located at . www.gutenberg.org https://www.gutenberg.org/policy/license.html

EXPRESSION and CHARACTER.

ATOMS, MOLECULES, AND ETHER WAVES

On the Economy of Machinery and Manufactures by Charles Babbage is part of the HackerNoon Books Series. You can jump to any chapter in this book . On Over Manufacturing here Chapter 24. On Over Manufacturing One of the natural and almost inevitable consequences of competition is the production of a supply much larger than the demand requires. This result usually arises periodically; and it is equally important, both to the masters and to the workmen, to prevent its occurrence, or to foresee its arrival. In situations where a great number of very small capitalists exist—where each master works himself and is assisted by his own family, or by a few journeymen—and where a variety of different articles is produced, a curious system of compensation has arisen which in some measure diminishes the extent to which fluctuations of wages would otherwise reach. This is accomplished by a species of middlemen or factors, persons possessing some capital, who, whenever the price of any of the articles in which they deal is greatly reduced, purchase it on their own account, in the hopes of selling at a profit when the market is better. These persons, in ordinary times, act as salesmen or agents, and make up assortments of goods at the market price, for the use of the home or foreign dealer. They possess large warehouses in which to make up their orders, or keep in store articles purchased during periods of depression; thus acting as a kind of flywheel in equalizing the market price. The effect of over-manufacturing upon great establishments is different. When an over supply has reduced prices, one of two events usually occurs: the first is a diminished payment for labour; the other is a diminution of the number of hours during which the labourers work, together with a diminished rate of wages. In the former case production continues to go on at its ordinary rate: in the latter, the production itself being checked, the supply again adjusts itself to the demand as soon as the stock on hand is worked off, and prices then regain their former level. The latter course appears, in the first instance, to be the best both for masters and men; but there seems to be a difficulty in accomplishing this, except where the trade is in few hands. In fact, it is almost necessary, for its success, that there should be a combination amongst the masters or amongst the men; or, what is always far preferable to either, a mutual agreement for their joint interests. Combination amongst the men is difficult, and is always attended with the evils which arise from the ill-will excited against any persons who, in the perfectly justifiable exercise of their judgement, are disposed not to act with the majority. The combination of the masters, on the other hand, is unavailing, unless the whole body of them agree, for if any one master can procure more labour for his money than the rest, he will be able to undersell them. If we look only at the interests of the consumer, the case is different. When too large a supply has produced a great reduction of price, it opens the consumption of the article to a new class, and increases the consumption of those who previously employed it: it is therefore against the interest of both these parties that a return to the former price should occur. It is also certain, that by the diminution of profit which the manufacturer suffers from the diminished price, his ingenuity will be additionally stimulated; that he will apply himself to discover other and cheaper sources for the supply of his raw material; that he will endeavour to contrive improved machinery which shall manufacture it at a cheaper rate; or try to introduce new arrangements into his factory, which shall render the economy of it more perfect. In the event of his success, by any of these courses or by their joint effects, a real and substantial good will be produced. A larger portion of the public will receive advantage from the use of the article, and they will procure it at a lower price; and the manufacturer, though his profit on each operation is reduced, will yet, by the more frequent returns on the larger produce of his factory, find his real gain at the end of the year, nearly the same as it was before; whilst the wages of the workman will return to their level, and both the manufacturer and the workman will find the demand less fluctuating, from its being dependent on a larger number of customers. It would be highly interesting, if we could trace, even approximately, through the history of any great manufacture, the effects of gluts in producing improvements in machinery, or in methods of working; and if we could shew what addition to the annual quantity of goods previously manufactured, was produced by each alteration. It would probably be found, that the increased quantity manufactured by the same capital, when worked with the new improvement, would produce nearly the same rate of profit as other modes of investment. Perhaps the manufacture of iron(1*) would furnish the best illustration of this subject; because, by having the actual price of pig and bar iron at the same place and at the same time, the effect of a change in the value of currency, as well as several other sources of irregularity, would be removed. At the present moment, whilst the manufacturers of iron are complaining of the ruinously low price of their produce, a new mode of smelting iron is coming into use, which, if it realizes the statement of the patentees, promises to reduce greatly the cost of production. The improvement consists in heating the air previously to employing it for blowing the furnace. One of the results is, that coal may be used instead of coke; and this, in its turn, diminishes the quantity of limestone which is required for the fusion of the iron stone. The following statement by the proprietors of the patent is extracted from Brewster's Journal, 1832, p. 349: Comparative view of the quantity of materials required at the Clyde iron works to smelt a ton of foundry pig-iron, and of the quantity of foundry pig-iron smelted from each furnace weekly Fuel in tons of 20 cwt each cwt 112 lbs; Iron-stone; Lime-stone
Cwt; Weekly produce in pig-iron Tons With air not heated and coke; 7;3 1/4; 15; 45 2. With air heated and coke; 4 3/4; 3 1/4; 10; 60 3. With air heated and coals not coked; 2 1/4; 3 1/4; 7 1/2; 65 Notes. 1. To the coals stated in the second and third lines, must be added 5 cwt of small coals, required to heat the air. The expense of the apparatus for applying the heated air will be from L200 to L300 per furnace. No coals are now coked at the Clyde iron works; at all the three furnaces the iron is smelted with coals. The three furnaces are blown by a double-powered steam-engine, with a steam cylinder 40 inches in diameter, and a blowing cylinder 80 inches in diameter, which compresses the air so as to carry 2 1/2 lbs per square inch. There are two tuyeres to each furnace. The muzzles of the blowpipes are 3 inches in diameter. The air heated to upwards of 600 degrees of Fahrenheit. It will melt lead at the distance of three inches from the orifice through which it issues from the pipe. The increased effect produced by thus heating the air is by no means an obvious result; and an analysis of its action will lead to some curious views respecting the future application of machinery for blowing furnaces. Every cubic foot of atmospheric air, driven into a furnace, consists of two gases.(2*) about one-fifth being oxygen, and four-fifths azote. According to the present state of chemical knowledge, the oxygen alone is effective in producing heat; and the operation of blowing a furnace may be thus analysed. The air is forced into the furnace in a condensed state, and, immediately expanding, abstracts heat from the surrounding bodies. Being itself of moderate temperature, it would, even without expansion, still require heat to raise it to the temperature of the hot substances to which it is to be applied. On coming into contact with the ignited substances in the furnace, the oxygen unites with them, parting at the same moment with a large portion of its latent heat, and forming compounds which have less specific heat than their separate constituents. Some of these pass up the chimney in a gaseous state, whilst others remain in the form of melted slags, floating on the surface of the iron, which is fused by the heat thus set at liberty. The effects of the azote are precisely similar to the first and second of those above described; it seems to form no combinations, and contributes nothing, in any stage, to augment the heat. The plan, therefore, of heating the air before driving it into the furnace saves, obviously, the whole of that heat which the fuel must have supplied in raising it from the temperature of the external air up to that of 600 degrees Fahrenheit; thus rendering the fire more intense, and the glassy slags more fusible, and perhaps also more effectually decomposing the iron ore. The same quantity of fuel, applied at once to the furnace, would only prolong the duration of its heat, not augment its intensity. The circumstance of so large a portion of the air(3*) driven into furnaces being not merely useless, but acting really as a cooling, instead of a heating, cause, added to so great a waste of mechanical power in condensing it, amounting, in fact, to four-fifths of the whole, clearly shews the defects of the present method, and the want of some better mode of exciting combustion on a large scale. The following suggestions are thrown out as likely to lead to valuable results, even though they should prove ineffectual for their professed object. The great difficulty appears to be to separate the oxygen, which aids combustion, from the azote which impedes it. If either of those gases becomes liquid at a lower pressure than the other, and if those pressures are within the limits of our present powers of compression, the object might be accomplished. Let us assume, for example, that oxygen becomes liquid under a pressure of 200 atmospheres, whilst azote requires a pressure of 250. Then if atmospheric air be condensed to the two hundredth part of its bulk, the oxygen will be found in a liquid state at the bottom of the vessel in which the condensation is effected, and the upper part of the vessel will contain only azote in the state of gas. The oxygen, now liquefied, may be drawn off for the supply of the furnace; but as it ought when used, to have a very moderate degree of condensation, its expansive force may be previously employed in working a small engine. The compressed azote also in the upper part of the vessel, though useless for combustion, may be employed as a source of power, and, by its expansion, work another engine. By these means the mechanical force exerted in the original compression would all be restored, except that small part retained for forcing the pure oxygen into the furnace, and the much larger part lost in the friction of the apparatus. The principal difficulty to be apprehended in these operations is that of packing a working piston so as to bear the pressure of 200 or 300 atmospheres: but this does not seem insurmountable. It is possible also that the chemical combination of the two gases which constitute common air may be effected by such pressures: if this should be the case, it might offer a new mode of manufacturing nitrous or nitric acids. The result of such experiments might take another direction: if the condensation were performed over liquids, it is possible that they might enter into new chemical combinations. Thus, if air were highly condensed in a vessel containing water, the latter might unite with an additional dose of oxygen, (4*) which might afterwards be easily disengaged for the use of the furnace. A further cause of the uncertainty of the results of such an experiment arises from the possibility that azote may really contribute to the fusion of the mixed mass in the furnace, though its mode of operating is at present unknown. An examination of the nature of the gases issuing from the chimneys of iron-foundries, might perhaps assist in clearing up this point; and, in fact, if such enquiries were also instituted upon the various products of all furnaces, we might expect the elucidation of many points in the economy of the metallurgic art. It is very possible also, that the action of oxygen in a liquid state might be exceedingly corrosive, and that the containing vessels must be lined with platinum or some other substance of very difficult oxydation; and most probably new and unexpected compounds would be formed at such pressures. In some experiments made by Count Rumford in 1797, on the force of fired gunpowder, he noticed a solid compound, which always appeared in the gunbarrel when the ignited powder had no means of escaping; and, in those cases, the gas which escaped on removing the restraining pressure was usually inconsiderable. If the liquefied gases are used, the form of the iron furnace must probably be changed, and perhaps it may be necessary to direct the flame from the ignited fuel upon the ore to be fused, instead of mixing that ore with the fuel itself: by a proper regulation of the blast, an oxygenating or a deoxygenating flame might be procured; and from the intensity of the flame, combined with its chemical agency, we might expect the most refractory ore to be smelted, and that ultimately the metals at present almost infusible, such as platinum, titanium, and others, might be brought into common use, and thus effect a revolution in the arts. Supposing, on the occurrence of a glut, that new and cheaper modes of producing are not discovered, and that the production continues to exceed the demand, then it is apparent that too much capital is employed in the trade; and after a time, the diminished rate of profit will drive some of the manufacturers to other occupations. What particular individuals will leave it must depend on a variety of circumstances. Superior industry and attention will enable some factories to make a profit rather beyond the rest; superior capital in others will enable them, without these advantages, to support competition longer, even at a loss, with the hope of driving the smaller capitalists out of the market, and then reimbursing themselves by an advanced price. It is, however, better for all parties, that this contest should not last long; and it is important, that no artificial restraint should interfere to prevent it. An instance of such restriction, and of its injurious effect, occurs at the port of Newcastle, where a particular Act of Parliament requires that every ship shall be loaded in its turn. The Committee of the House of Commons, in their Report on the Coal Trade, state that, 'Under the regulations contained in this Act, if more ships enter into the trade than can be profitablv employed in it, the loss produced by detention in port, and waiting for a cargo. which must consequently take place, instead of falling, as it naturally would, upon particular ships, and forcing them from the trade, is now divided evenly amongst them; and the loss thus created is shared by the whole number.' Report, p. 6. It is not pretended, in this short view, to trace out all the effects or remedies of over-manufacturing; the subject is difficult, and, unlike some of the questions already treated, requires a combined view of the relative influence of many concurring causes. NOTES: The average price per ton of pig iron, bar iron, and coal, together with the price paid for labour at the works, for a long series of years, would be very valuable, and I shall feel much indebted to anyone who will favour me with it for any, even short, period. The accurate proportions are, by measure, oxygen 21, azote 79. A similar reasoning may be applied to lamps. An Argand burner, whether used for consuming oil or gas, admits almost an unlimited quantity of air. It would deserve enquiry, whether a smaller quantity might not produce greater light; and, possibly, a different supply furnish more heat with the same expenditure of fuel. Deutoxide of hydrogen, the oxygenated water of Thenard. About HackerNoon Book Series: We bring you the most important technical, scientific, and insightful public domain books. This book is part of the public domain. Charles Babbage (2003). On the Economy of Machinery and Manufactures. Urbana, Illinois: Project Gutenberg. Retrieved https://www.gutenberg.org/cache/epub/4238/pg4238.html This eBook is for the use of anyone anywhere at no cost and with almost no restrictions whatsoever. You may copy it, give it away or re-use it under the terms of the Project Gutenberg License included with this eBook or online at , located at . www.gutenberg.org https://www.gutenberg.org/policy/license.html

On Over Manufacturing

ONCE AND NOW.

GATHERING CLOUDS.

Scientific American Supplement, No. 717, September 28, 1889, by Various, is part of the HackerNoon Books Series. You can jump to any chapter in this book . THE ELECTRIC AGE. here THE ELECTRIC AGE. By Charles Carleton Coffin. The application of electricity for our convenience and comfort is one of the marvels of the age. Never in the history of the world has there been so rapid a development of an occult science. Prior to 1819 very little was known in regard to magnetism and electricity. During that year Oersted discovered that an electric current would deflect a magnetic needle, thus showing that there was some relationship between electric and magnetic force. A few months later, Arago and Sir Humphry Davy, independently of each other, discovered that by coiling a wire around a piece of iron, and passing an electric current through it, the iron would possess for the time being all the properties of a magnet. In 1825 William Sturgeon, of London, bent a piece of wire in the form of the letter U, wound a second wire around it, and, upon connecting it with a galvanic battery, discovered that the first wire became magnetic, but lost its magnetic property the moment the battery was disconnected. The idea of a telegraphic signal came to him, but the electric impulse, through his rude apparatus, faded out at a distance of fifty feet. In 1830 Prof. Joseph Henry, of this country, constructed a line of wire, one and a half miles in length, and sent a current of electricity through it, ringing a bell at the farther end. The following year Professor Faraday discovered magnetic induction. This, in brief, is the genesis of magnetic electricity, which is the basis of all that has been accomplished in electrical science. The first advance after these discoveries was in the development of the electric telegraph—the discovery in 1837, by the philosopher Steinhill, that the earth could serve as a conductor, thus requiring but one wire in the employment of an electric current. Simultaneously came Morse's invention of the mechanism for the telegraph in 1844, foreshadowed by Henry in the ringing of bells, thus transmitting intelligence by sound. Four years later, in 1848, Prof. M. G. Farmer, still living in Eliot, Me., attached an electro-magnet to clockwork for the striking of bells to give an alarm of fire. The same idea came to William F. Channing. The mechanism, constructed simply to illustrate the idea by Professor Farmer, was placed upon the roof of the Court House in Boston, and connected with the telegraph wire leading to New York, and an alarm rung by the operator in that city. The application of electricity for giving definite information to firemen was first made in Boston, and it was my privilege to give the first alarm on the afternoon of April 12, 1852. At the close of the last century, Benjamin Thompson, born in Woburn, Mass., known to the world as Count Rumford, was in the workshop of the military arsenal of the King of Bavaria in Munich, superintending the boring of a cannon. The machinery was worked by two horses. He was surprised at the amount of heat which was generated, for when he threw the borings into a tumbler filled with cold water, it was set to boiling, greatly to the astonishment of the workmen. Whence came the heat? What was heat? The old philosopher said that it was an element. By experiment he discovered that a horse working two hours and twenty minutes with the boring machinery would heat nineteen pounds of water to the boiling point. He traced the heat to the horse, but with all his acumen he did not go on with the induction to the hay and oats, to the earth, the sunshine and rain, and so get back to the sun. One hundred years ago there was no chemical science worthy of the name, no knowledge of the constitution of plants or the properties of light and heat. The old philosophers considered light and heat to be fluids, which passed out of substances when they were too full. Count Rumford showed that motion was convertible into heat, but did not trace the motion to its source, so far as we know, in the sun. It is only forty-six years since Professor Joule first demonstrated the mutual relations of all the manifestations of nature's energy. Thirty-nine years only have passed since he announced the great law of the convertibility of force. He constructed a miniature churn which held one pound of water, and connected the revolving paddle of the churn with a wheel moved by a pound weight, wound up the weight, and set the paddle in motion. A thermometer detected the change of temperature and a graduated scale marked the distance traversed by the descending weight. Repeated experiments showed that a pound weight falling 772 feet would raise the temperature of water one degree, and that this was an unvarying law. This was transferring gravitation to heat, and the law held good when applied to electricity, magnetism, and chemical affinity, leading to the conclusion that they were severally manifestations of one universal power.— Congregationalist. About HackerNoon Book Series: We bring you the most important technical, scientific, and insightful public domain books. This book is part of the public domain. Various (2006). Scientific American Supplement, No. 717, September 28, 1889. Urbana, Illinois: Project Gutenberg. Retrieved https://www.gutenberg.org/cache/epub/17755/pg17755-images.html This eBook is for the use of anyone anywhere at no cost and with almost no restrictions whatsoever. You may copy it, give it away or re-use it under the terms of the Project Gutenberg License included with this eBook or online at , located at . www.gutenberg.org https://www.gutenberg.org/policy/license.html

THE ELECTRIC AGE

We held a council, to decide on our future movements

TWO DAYS BEFORE THE TRIAL.

Stories, News & Mentions about RUFFO

Ruffo’s Stories on HackerNoon