Numerade is a venture-backed, high-growth education technology startup based in Los Angeles

Numerade

numerade

The Handy Guide to Getting Google's Bard AI to List Your Brand or Business In Its Response

HackerNoon Reporter: Please tell us briefly about your  background. I’m a data scientist. My focus is causal inference. Thematically, I’m most interested in political violence and organized crime. I had always been interested in the intersection of money, crime, and politics. I received a solid qualitative and quantitative foundation at Georgetown University’s Center for Peace and Security Studies, and NYU’s Department of Politics in their doctoral program, respectively. Somewhere in between, I spent time consulting at the Defense Advanced Research Projects Agency (DARPA) on the disciplined use of data during active wars. These experiences led me to where I am today, serving as president of a cutting-edge and innovative company that builds machine learning software for intelligence, compliance, and security professionals. What's your startup called? And in a sentence or two, what does it do? Our startup is called Giant Oak. At Giant Oak, we use an understanding of economics, behavioral science, and machine learning technology to enhance screening and vetting processes and stop illicit behavior like money laundering, human trafficking, and fraud. Our flagship product, GOST® (Giant Oak Search Technology), is a trusted tool that enables businesses in any industry to screen and continuously monitor large sets of data and companies quickly and efficiently. What is the origin story? While working in the financial and government spaces, we saw a need across industries for enhanced entity screening processes that use AI and machine learning to identify bad actors. In particular, we noticed a dearth of tools that make use of publicly available information; this just did not make sense to us. The largest trove of information out there in the world is sitting there underutilized, and most providers in the space were (and still are) just reselling databases. With GOST, we quickly and empirically proved that prior processes were inefficient and ineffective. Every day, human traffickers, fraudsters, and money launderers were getting away with crimes that GOST could help users identify. What do you love about your team, and why are you the ones to solve this problem? Our Giant Oak team is driven by the mission to make the world safer by building trusted, world-changing tools that reduce crime, fraud, and violence. That allows us to step back, scan how the market is doing things, make critical assessments about what is and is not working, and fix glaring problems. We’re not bound by any particular data or technology. We do, however, hold ourselves accountable to five core values that set our culture apart - be kind, be respectful, be smart, be bold, and be scientific. If you weren’t building your startup, what would you be doing? I teach data science to security professionals, and I’m eager to build and grow that curriculum. There is a great need today for policymakers to be more numerate to make data-driven decisions and ask the right questions of analytic products. I’d love to be instrumental in training and mentoring a generation of brilliant decision-makers who can engage critically and thoughtfully with data. At the moment, how do you measure success? What are your core metrics? Like most organizations, we measure success through revenue, growth, and real-world impact. We do, however, put a considerable amount of emphasis on real-world impact. What’s most exciting about your traction to date? Our tool is more than an idea; it’s a proven solution that has a real-life impact. Every day our clients, both financial and government, send us notes about the bad actors they’ve identified. It’s amazing to see that these organizations can use GOST and stop human traffickers, fraudsters, money launderers, terrorists, and other criminals every day. There’s nothing more rewarding than knowing that your tool is actually helping the world become a safer place. What technologies are you currently most excited about, and most worried about? And why? I am most excited about aspects of machine learning that enable increased performance while preserving privacy. Right now, those technologies are enabled by federated learning and transfer learning. What I’m most concerned about is the aggregation of databases, which risks undermining privacy. If we don’t embrace new technologies that do preserve privacy as technology advances, that is worrying. What drew you to get published on HackerNoon? What do you like most about our platform? We were thrilled to be nominated for the 2021 Startup of the Year by Hacker Noon. It means a lot when platforms support and recognize a community of startups that are striving to make a difference in the world. What advice would you give to the 21-year-old version of yourself? Walkthrough and explore the space behind every open door instead of waiting for particular doors to open. What is something surprising you've learned this year that your contemporaries would benefit from knowing? The biggest lesson I learned this year, as president of a company and a team of dedicated employees, is that having a nice office isn’t the perk that it used to be, even just a year ago. Working from anywhere is now the perk that employees prefer; the thing to focus on now is not where your employees sit but rather building cohesion and culture remotely. Vote for Giant Oak as the startup of the year, Arlington.

Figthing the Bad Actors with Machine Learning: Startup Interview with Harsh Pandya, CEO Giant Oak

A is a symbol or group of symbols that represents a number. The symbols “11(decimal)”, “1011(binary)” and “B(hexadecimal)” are numerals, all representing the number. The number is the idea, the numeral is anything that represents that idea. numeral Numerals are not the same as numbers just as words are not the same with the things they refer to. different same A positional numeral system denotes usually the extension to any of the (or ). In a more general sense, a positional system is a numeral system in which the to the value of a number is the . base Hindu–Arabic numeral system decimal system contribution of a digit value of the digit multiplied by a factor determined by the position of the digit In modern positional systems, such as the , the of the digit means that its value must be multiplied by some value: in 555, the three identical symbols represent five , five , and five , respectively, due to their different in the digit string. decimal system position hundreds tens units positions According to its of occurrence in the number, each digit is . Towards the left, the weights increase by a constant factor equivalent to the or . With the help of the radix point (‘.’), the positions corresponding to integral weights (1) are differentiated from the positions corresponding to fractional weights (<1). position weighted base radix Any integer value that is greater than or equal to two can be used as the base or . radix The digit position ’ ’ has weight . . n r ^n The largest value of digit position is always 1 less than the base value The value of a number is a weighted sum of its digits. For example: Decimal (0–9 digits with radix 10) 2056 value breakdown Hexadecimal(0–9 & A-F digits with radix 16) 171B value breakdown The general form of value breakdown, where b is the base or radix of the numeral system The Idea to package Quick Link I wanted to be able to and play around with without having to worry about the semantics of each system. At the end of the day, as we mentioned before can represent the . So even operations between numerals from different systems shouldn’t be an issue when they can be translated to represent the same numbers. implement any numeral system the representation of numbers multiple different numerals same number For some reason, an image from the movie flashed in my head. The where machine, the , came to life and the mechanical rotors started spinning. Imitation Game beloved moment in human history Alan Turing’s bombe A digit in any positional numeral system can be by the it can take. Each state has a and a . The of the is the while producing a . represented possible states next state previous state next state last state first state carry-over A data structure like a can effectively represent what we mentioned in the above paragraph. circular list , . In a doubly-linked circular list the next pointer of the last node points to the first node, and the previous pointer of the first node points to the last node making the circular in both directions Here comes Go’s . With the ring, we can easily represent of by filling it with the possible ordered states and an initial value. ring any single digit any positional numeral system { r := ring.New( (values)) _, e := values {
      r.Value = e
      r = r.Next()
   } indexOf(state, values) == { , fmt.Errorf( , state, values)
   } values { r.Value == state { }
      r = r.Next()
   } r, } // newDigit creates and initializes a new digit (ring). func newDigit (values [] , state ) rune rune (*ring.Ring, error) // initialize a new empty ring len // fill the ring with values for range if -1 return nil "invalid digit. value: %v does not exist in possible values: %v" // roll the ring in desired "state" position. for range if break return nil What happens when a numeral contains more than one digit? That can easily be solved by having a whose elements would be consisted of doubly-linked list doubly-linked circular list s. This way we can easily create positional representation of any number. any { number := Numeral{
      digits:      list.New(),
      digitValues: values,
   } i := ; i < (initial); i++ {
      digit, err := newDigit(values, (initial[i])) err != { , err
      }
      number.digits.PushBack(digit)
   } &number, } // NewNumeral initializes a numeral by providing the initial number in strings // along with the possible values that each digit can have. func NewNumeral (values [] , initial ) rune string (*Numeral, error) // initialise a new number. // add digits to the number along with their state. for 0 len rune if nil return nil return nil Every time a resets back to the “first” state it would trigger a to the next most valuable digit. Pretty simple right? digit Next() In the case that our digits are all full and we want to increment, we can just create a new ring, initialize it and link it to the left side of our list. Think of an example, a hypothetical numeral system with which has a , and also imagine that we want to have a 4 “digit numeral” like “2z3r”. symbols [0–9,a-z] radix of 36 Numeral {
   digits      *list.List
   digitValues [] } // Numeral represents a numeral that is consisted by its digits // and digit values. type struct rune By defining a number like this all you need to have is a you want to your number . slice of the symbols represent in the order they will have to exist values= [] { , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , } var rune '0' '1' '2' '3' '4' '5' '6' '7' '8' '9' 'a' 'b' 'c' 'd' 'e' 'f' 'g' 'h' 'i' 'j' 'k' 'l' 'm' 'n' 'o' 'p' 'q' 'r' 's' 't' 'u' 'v' 'w' 'x' 'y' 'z' Then you need a “constructor” to generate numerals at will. { number := Numeral{
      digits:      list.New(),
      digitValues: values,
   } i := ; i < (initial); i++ {
      digit, err := newDigit(values, (initial[i])) err != { , err
      }
      number.digits.PushBack(digit)
   } &number, } // NewNumeral initializes a numeral by providing the initial number in strings // along with the possible values that each digit can have. func NewNumeral (values [] , initial ) rune string (*Numeral, error) // initialise a new number. // add digits to the number along with their state. for 0 len rune if nil return nil return nil Initializing a numeral after we have reached this point is pretty straightforward. number, _ := numeral.NewNumeral(values, ) "2z3r" Performing Simple Operations This approach makes +1 and -1 operations since always know their next and previous node. very fast lists (either circular or not) For example: Incrementing a numeral by needs to execute The result is “12d5”. Hexadecimal “12d4” one 1 next() to the rightmost ring, that's all. Incrementing a numeral “ ” by needs to execute . The result is “e000”. Hexadecimal dfff one 4 next() calls on the 3 rightmost rings and one last on the 4th most significant ring Decrementing1will operate in a similar way. { e := n.digits.Back(); e != ; e = e.Prev() { r := e.Value.(*ring.Ring) r = r.Next()
      e.Value = r r.Value != n.digitValues[ ] { } e.Prev() == {
         d, _ := newDigit(n.digitValues, n.digitValues[ ])
         n.digits.PushFront(d)
      }
   } } // Increment performs a +1 to the Numeral. func (n *Numeral) Increment () error // take the first digit from the right and keep going if there are any arithmetic holdings. for nil // get current ring. // rotate and update. // if the digit is not being reset (no arithmetic holdings) then there is no need to // proceed in adding on the others. if 0 break // If needed add an extra new digit on the left side. if nil 0 return nil Adding and Subtracting can also happen efficiently even between numerals that are based on different systems by using the most known numbers of the decimal system that our computers easily understand. will just require both of those to , add them and generate a new numeral from the . The result numeral can even be on a different numeral system easily by generating it from an integer. Adding 2 numerals converting decimal integers result Converting any numeral to integer is easy by performing a value breakdown {
   dec := di := d := n.digits.Back(); d != ; d = d.Prev() { r := d.Value.(*ring.Ring) i := indexOf(r.Value.( ), n.digitValues) dec = dec + i*powInt( (n.digitValues), di)
      di++
   } dec
} // Decimal converts a numeral to a decimal integer. func (n *Numeral) Decimal () int 0 0 for nil // get current ring. // get the index of the ring. rune // Add digit's decimal counterpart to the decimal sum. len return Let’s Talk Numbers Running some on the method for for different initial values (small and big) results in some very nice results. The average execution time of the Increment function for binary numerals after 1 billion increments is around 7.6ns / op. benchmarks Increment() 3 different numeral systems As one may expect, therefore the transitions that need to happen are (increments/decrements)which results in better performance. selecting a numeral system with a higher numeral range results in carryovers appearing less frequently, fewer goos: linux
goarch: amd64
pkg: github.com/slysterous/numeral
cpu: Intel(R) Core(TM) i7– CPU @ GHz
BenchmarkNumeralIncrementDecimalSmall ns/op
BenchmarkNumeralIncrementDecimalLarge ns/op
BenchmarkNumeralIncrementBinarySmall ns/op
BenchmarkNumeralIncrementBinaryLarge ns/op
BenchmarkNumeralIncrementHexSmall ns/op
BenchmarkNumeralIncrementHexLarge ns/op 7700 3.60 -8 1000000000 4.604 -8 1000000000 4.574 -8 1000000000 7.644 -8 1000000000 7.655 -8 1000000000 4.427 -8 1000000000 4.652 How about converting numerals into decimal integers? From the benchmarks that I performed on Hex and Binary numerals, the process takes . The opposite process, converting from decimal ints to any numeral is similar. less than 1 μsecond Since converting to decimals , it’s easy to reach the conclusion that , . This can also be reflected in the below benchmark. requires traversing through every digit to sum its weight the more digits there are the slower the process will be go test -bench=NumeralDecimal -benchtime= s
goos: linux
goarch: amd64
pkg: github.com/slysterous/numeral
cpu: Intel(R) Core(TM) i7– CPU @ GHz
BenchmarkNumeralDecimalFromHex0– ns/op
BenchmarkNumeralDecimalFromHexfffff ns/op
BenchmarkNumeralDecimalFromHexffffffffff ns/op
BenchmarkNumeralDecimalFromHexffffffffffffffffffff ns/op
BenchmarkNumeralDecimalFromBinary0– ns/op
BenchmarkNumeralDecimalFromBinary10000– ns/op
BenchmarkNumeralDecimalFromBinary1000000000– ns/op
BenchmarkNumeralDecimalFromBinary1000000000000000000– ns/op 60 7700 3.60 8 1000000000 8.182 -8 252367634 284.8 -8 250184980 286.0 -8 100000000 616.6 8 1000000000 7.818 8 837013039 84.98 8 337445121 214.8 8 158028769 455.9 Alright, one last check. How does adding numerals of different systems perform? Remember, I’m using decimal integers as the intermediate common base to perform operations. Turns out adding is also relatively fast since it to perform an average addition takes roughly around 8 μseconds Additions are a bit trickier in terms of performance. At least 2 conventions have to happen, both numbers are translated to decimal and in the end, the result is converted back to the final desired numeral system. This on the . relies heavily performance of the conversion of a numeral to decimal int go test -bench=NumeralAdd -benchtime= s
goos: linux
goarch: amd64
pkg: github.com/slysterous/numeral
cpu: Intel(R) Core(TM) i7– CPU @ GHz
BenchmarkNumeralAddBinaryOnHex ns/op
BenchmarkNumeralAddHexOnHex ns/op
BenchmarkNumeralAddDecOnDec ns/op
BenchmarkNumeralAddSumBinaryOnHex ns/op
BenchmarkNumeralAddSumHexOnHex ns/op
BenchmarkNumeralAddSumDecOnDec ns/op 60 7700 3.60 -8 9691966 8651 -8 9144648 8290 -8 11384712 7014 -8 11281405 6237 -8 12745245 5873 -8 12672326 5849 In conclusion, is now a Go package. It gives you the ability to create custom (positional) numeral systems and perform operations on them. Feel free to contribute and don’t hesitate to reach out for any questions. numeral Also published on Medium's slysterous

Introduction to Numeral Systems in Go

La paginación es una técnica ampliamente utilizada en el diseño web que divide el contenido en varias páginas, presentando grandes conjuntos de datos de una manera mucho más fácil de digerir para los internautas. Hay muchos métodos de paginación empleados por diferentes desarrolladores web, como la paginación numerada, el desplazamiento infinito, etc. Aunque generalmente se cree que la paginación mejora la experiencia del usuario, la mala noticia es que dificulta el web scraping. Si está tratando de scrape datos de un sitio web y enfrenta un dilema sobre cómo abordar la paginación para el web scraping, lo tenemos cubierto. , una herramienta automática de web scraping, admite el manejo de sitios web de varias estructuras de paginación. Ahora vamos a ilustrar los diversos enfoques sobre cómo lidiar con diferentes tipos de paginación con Octoparse, que incluye: Octoparse Paginación con "next” botón Número de página sin botón "Next" Desplazamiento infinito "Load More" Botón 1. Paginación con next botón Hacer clic en el botón ”Siguiente" para paginar es quizás uno de los métodos más utilizados para la paginación, lo que facilita a los visitantes navegar por las páginas del sitio web. Es muy sencillo manejar este tipo de paginación para el web scraping en Octoparse. No importa si se trata de un botón siguiente que se muestra en la forma de la palabra - "Next" o simplemente una flecha hacia la derecha - ">", solo necesita crear un "bucle de paginación” para continuar haciendo clic en la página del botón después de scrape el objeto actual. (Mira un ejemplo ) aquí 2. Número de página sin botón "Siguiente" El enfoque para este tipo particular de paginación es muy similar al del botón siguiente. Desea crear un ciclo de paginación para seguir haciendo clic en el siguiente número de página en la línea. Sin embargo, dado que no hará clic en un elemento estático, localizar el siguiente número de página con precisión es fundamental. Octoparse usa XPath (Lenguaje de XML Path, que usa la sintaxis "path like" para identificar y navegar por los nodos en un documento XML) para localizar cualquier elemento. Entonces, el punto clave aquí es modificar la XPath del "buclo de paginación” para asegurarse de que siempre localizará el siguiente número de página tan pronto como la página actual se haya eliminado por completo (consulte para saber cómo modificar la XPath para ubicar con precisión la siguiente página número) este tutorial 3. Desplazamiento infinito Infinite-scrolling, also know as "endless scrolling" is a technique used most often by websites with JavaScript or AJAX to load additional content dynamically as users scroll down to the bottom of the webpage. Instead of using "previous/next" pagination buttons, many websites are turning to infinite scrolling, saving people from having to click through the many pages. Infinitive scrolling is typically used by websites with a large amount of data to display such as social media platforms like Facebook and Twitter. El desplazamiento infinito, también conocido como "endless scrolling", es una técnica utilizada con mayor frecuencia por los sitios web con JavaScript o AJAX para cargar contenido adicional de forma dinámica a medida que los usuarios se desplazan hacia la parte inferior de la página web. En lugar de utilizar los botones de paginación "anterior/siguiente", muchos sitios web están recurriendo al "desplazamiento infinito", evitando que las personas tengan que hacer clic en las muchas páginas. El desplazamiento infinitivo generalmente es utilizado por sitios web con una gran cantidad de datos para mostrar, como plataformas de redes sociales como Facebook y Twitter. Octoparse se ocupa del infinitive scrolling imitando el comportamiento del desplazamiento. Dependiendo de la cantidad de contenido que desee cargar, simplemente configure el tiempo de desplazamiento y la forma de desplazamiento adecuados, la página se desplazará automáticamente (consulte un ejemplo ) aquí 4. "Load More" Botón Cargar más botones de navegación es otra alternativa popular al desplazamiento infinito. En este caso, tendría un botón específico, como "Cargar más", para activar la carga de contenido con AJAX cuando llegue al final de la página. Octoparse trata con el botón "Cargar más" con un ciclo de paginación, que es lo mismo que tratamos con el botón "Siguiente", haciendo clic en un solo botón repetidamente. Sin embargo, la diferencia es que con el botón "Cargar más", debemos ejecutar el ciclo de paginación hasta que desaparezca el botón de carga antes de continuar con el siguiente paso. Después de cargar todo el contenido deseado, el proceso de scraping es tan fácil como scrape una sola página (verifique más detalles ) aquí Conclusion Pagination reduces page complexity and improves the readability of web content, yet it needs to be tackled using various approaches, whichever that creates maximum efficiency. If we fail to deal with pagination properly, it will result in missing data and a waste of time. Making good use of a web scraping tool like , you can avoid the complexities of web scraping! Octoparse Previously published at https://www.octoparse.es/blog/abordar-la-paginacion-para-web-scraping

Web Scraping 101: Abordar La Paginación para Web Scraping

When you inject new software into an environment, it’s difficult to forecast what will transpire. Product teams continually try to predict what will work. We predict that a certain new feature will increase conversion, that a targeted ad spend will bring critical mass, or that market conditions will change in our favor. We expect many of our bets to fail. We inoculate ourselves from bad predictions with methodologies like Agile and Lean Startup. Measuring market signal with each iteration helps us quickly change course. If the brilliant new feature decreases conversion, we can remove it before our revenue plummets. Minimizing losses from bad bets give us more opportunities to try things. However, the experience of failed attempts does not reliably lead to better predictions in the future. Faced with new information, we can overreact, under-react, misunderstand, forget, or turn a blind eye. Emotions and cognitive biases can cloud our judgment. Learning can go even more haywire at an organizational level. Add group-think, knowledge silos, and to the list of epistemological threats. corporate amnesia But imagine if we could systematically improve the accuracy of our company’s predictions. Then we could reliably increase our ability to make winning bets over time. I was surprised to discover that a playbook for optimizing prediction accuracy already exists. It’s called Superforecasting. But it’s not being widely leveraged by product teams yet. Superforecasting The first step to improving something is measuring it. In the book , Philip E. Tetlock and Dan Gardner describe a system for measuring the accuracy of predictions called the Brier score. In their words, “Brier scores measure the distance between what you forecast and what actually happened.” Superforecasting To explain Brier scores, let’s consider weather forecasting. Weather forecasting involves making probabilistic predictions like “There’s a 70% chance it will rain tomorrow.” If it doesn’t rain tomorrow, it doesn’t mean the forecast was wrong since it gave a 30% chance of no rain. This can be counter-intuitive since people have a false tendency to think that a forecast is wrong when a prediction that something is likely doesn’t end up happening. In the terminology of the Brier score, a forecaster’s predictions are “well calibrated” if the things they say are X% likely to happen actually happen X% of the time. So if a meteorologist says “there’s 70% chance of rain tomorrow” one hundred times during the year, they would be perfectly calibrated if it rained 70 times and didn’t rain the other 30. If it had only rained 50 times, we would say the forecaster is over-confident. If it had rained 80 times, they’re under-confident. Over-confidence and under-confidence are the two forms of mis-calibration. Under-confidence Perfect calibration The other dimension of the Brier score is resolution. Resolution measures the decisiveness of a forecaster’s predictions. If a meteorologist says the chance of rain is 100% when it does rain and 0% when it doesn’t, they would have perfect resolution. Forecasters can be perfectly calibrated with poor resolution. Let’s say a forecaster predicts a 50% chance of rain every day. Even if it does rain exactly half of the days, their resolution will be dinged for not elevating the chances on the days it does rain and lowering them on the days when it doesn’t. Poor resolution implies overly cautious predictions. The Brier score combines calibration and resolution into a single accuracy metric. The lower the Brier score, the more accurate a forecaster’s predictions. With a metric for prediction accuracy, we can scientifically investigate which factors lead individuals or teams to improve their foresight. Tetlock did exactly this to win a forecasting tournament administered by the United States Intelligence Community. The forecasting tournament was created in the wake of the tragic false assessment that Iraq possessed WMDs. According to the , the goal of the government program is to IARPA website … dramatically enhance the accuracy, precision, and timeliness of intelligence forecasts for a broad range of event types, through the development of advanced techniques that elicit, weight, and combine the judgments of many intelligence analysts. The tournament, held over four years, asked participants to forecast questions like “Will Russia officially annex additional Ukrainian territory in the next three months?” and “In the next year, will any country withdraw from the eurozone?” As measured by the Brier score, the predictions made by Tetlock’s group of “superforecasters” were 30% more accurate than the average for intelligence community analysts who has access to classified information. While the highest performing forecasters are smart, Tetlock concluded that making consistently accurate predictions has more to do with they think. Tetlock’s discoveries constitute a set of proven principles of good judgment, summarized here: how In philosophic outlook, [superforecasters] tend to be: CAUTIOUS: Nothing is certain HUMBLE: Reality is infinitely complex NONDETERMINISTIC: What happens is not meant to be and does not have to happen In their abilities and thinking styles, they tend to be: ACTIVELY OPEN-MINDED: Beliefs are hypotheses to be tested, not treasures to be protected INTELLIGENT AND KNOWLEDGEABLE, WITH A “NEED FOR COGNITION”: Intellectually curious, enjoy puzzles and mental challenges REFLECTIVE: Introspective and self-critical NUMERATE: Comfortable with numbers In their methods of forecasting they tend to be: PRAGMATIC: Not wedded to any idea or agenda ANALYTICAL: Capable of stepping back from the tip-of-your-nose perspective and considering other views DRAGONFLY-EYED: Value diverse views and synthesize them into their own PROBABILISTIC: Judge using many grades of maybe THOUGHTFUL UPDATERS: When facts change, they change their minds GOOD INTUITIVE PSYCHOLOGISTS: Aware of the value of checking thinking for cognitive and emotional biases In their work ethic, they tend to have: A GROWTH MINDSET: Believe it’s possible to get better GRIT: Determined to keep at it however long it takes I love how Tetlock’s research puts data to philosophically rich concepts, like the . The Ancient Greek poet Archilochus famously said that “a fox knows many things, but a hedgehog one important thing.” Tetlock categorized his forecasters into each persona. hedgehog-fox dichotomy Hedgehogs are fixated on one Big Idea; a single world view. They see everything through that lens. Tetlock describes this group in his forecasters pool: Some were environmental doomsters (“ We’re running out of everything”); others were cornucopian boomsters (“ We can find cost-effective substitutes for everything”). Some were socialists (who favored state control of the commanding heights of the economy); others were free-market fundamentalists (who wanted to minimize regulation). As ideologically diverse as they were, they were united by the fact that their thinking was so ideological. They sought to squeeze complex problems into the preferred cause-effect templates and treated what did not fit as irrelevant distractions. Allergic to wishy-washy answers, they kept pushing their analyses to the limit (and then some), using terms like “furthermore” and “moreover” while piling up reasons why they were right and others wrong. As a result, they were unusually confident and likelier to declare things “impossible” or “certain.” Foxes, in contrast, are open to many perspectives. They synthesize inputs from multiple models. Tetlock writes: The [fox] group consisted of more pragmatic experts who drew on many analytical tools, with the choice of tool hinging on the particular problem they faced. These experts gathered as much information from as many sources as they could. When thinking, they often shifted mental gears, sprinkling their speech with transition markers such as “however,” “but,” “although,” and “on the other hand.” They talked about possibilities and probabilities, not certainties. And while no one likes to say “I was wrong,” these experts more readily admitted it and changed their minds. The hedgehog vs. fox battle has been officially settled: foxes win, at least as far as forecasting goes. Tetlock found that “Foxes beat hedgehogs on both calibration and resolution. Foxes had real foresight. Hedgehogs didn’t.” Imagine an economist who relentlessly preaches austerity in the face of clear evidence showing the failure of austerity measures. Yet, Tetlock doesn’t close the door on the utility of the hedgehog mindset. He raises the possibility that, while Hedgehogs suck at forecasting, their commitment to a single frame of thinking might lead them to ask more illuminating questions. Tetlock speculates: The psychological recipe for the ideal superforecaster may prove to be quite different from that for the ideal superquestioner, as superb question generation often seems to accompany a hedgehog-like incisiveness and confidence that one has a Big Idea grasp of the deep drivers of an event. That’s quite a different mindset from the foxy eclecticism and sensitivity to uncertainty that characterizes superb forecasting. This is a compelling hypothesis for how foxes and hedgehogs can collaborate to achieve great things. Applying Superforecasting to Startups Now that we’ve taken a quick tour of superforecasting, let’s consider how the system could be applied to product development. Imagine your team at an eCommerce startup is finishing a project to redesign the checkout page. Right before you launch an A/B test with the new page, you pose the question: “Will the purchase conversion rate of the new checkout page beat the current page by 10% for the 30 days after launch?” Everyone on your team provides probabilistic forecasts ranging from 10% to 80%. The average prediction gives a 30% chance that the new page will increase conversion by 10%. If you had seen these forecasts you committed to doing the project, would you have moved forward with it? before It depends. Given the scale of your company, maybe a 30% chance to increase revenue by 10% is a good bet to make. But maybe other endeavors are more likely to succeed, so you start forecasting all proposed projects before you prioritize them. And your forecasts need not be limited to the impact of potential projects. You can predict other factors related to the inner workings of your company, the technological landscape, or the market. Some examples: “Will the engineering team finish refactoring our accounting system within two months of the project start date?” “If we added 5 employees to the sales team in March, will our customer base grow by 20% by the end of the year?” “Will the augmented/virtual reality market exceed $100 billion in revenue by 2020?” “Will the word recognition accuracy rate of Google Voice reach 99% by the end of 2019?” “Will the average health insurance deductible increase by 5% by the end of 2018?” With accurate forecasts to questions like these, we’ll be more aware of our capabilities and steps ahead of the market. We’d have the foresight necessary for exceptional strategy. One might argue that companies are already making forecasts in the form of cost-benefit analysis or financial models. But these formats are too ambiguous to help us improve our predictions over time. Tetlock and Gardner write: The first step in learning what works in forecasting, and what doesn’t, is to judge forecasts, and to do that we can’t make assumptions about what the forecast means. We have to know. There can’t be any ambiguity about whether a forecast is accurate or not… In cost-benefit analysis, you might forecast the impact of a project to be high. But what exactly does “high” mean? And in what time frame must the benefit occur? There are too many shades of gray. And maybe we’re two months after launch with little impact from the project. Does that mean the prediction is wrong or do we give it more time? With a financial model, the projections will surely not be right. So how can we determine if the model was correct? How close do the numbers need to be? exactly Cost-benefit analysis and financial models serve a real purpose, but they must be converted into unambiguous forecasts if we’re serious about improving our predictions over time. So let’s say our company starts making a high quantity of unambiguous predictions. If we calculated our collective Brier score, we’d probably see that it leaves much to be desired. Besides the fact that accurate forecasting is hard, I suspect many startup people are especially poor forecasters. There’s a hedgehoggy aspect of working on a product. It requires an almost irrational devotion to the product direction; a narrow lens that misses relevant opposing perspectives. We may be prone to overconfidence in predicting the success of product launches or improvement of their environmental conditions. Exuberant optimism is conducive to good work but runs counter to accurate forecasting. Perhaps company staff is too biased to proficiently forecast their own future. The solution could be to insert 3rd party superforecasters into a company’s operations. It appears that based on this idea. I’m compelled to try this out. In the future, asking questions of superforecasters could become ubiquitous, like running user tests or customer surveys. I anticipate there’d be some challenges in transferring domain knowledge, at least as far as some key questions are concerned. But this can be overcome. Tetlock co-founded a commercial firm But even without enlisting external superforecasters, applying the model is useful. These practices will be beneficial no matter what: Creating unambiguous hypotheses for projects Making predictions before you commit to resourcing Looping back to old predictions to judge their accuracy Turning these practices into habits is the goal of my side-project, . Double-Loop A good way to summarize the potential of the superforecasting model for product teams is that it nudges us to become more like , a hybrid animal invented by Venkatesh Rao. He explains: foxhogs The net effect of this personality profile is that the foxhog can live in a highly uncertain environment, and be responsively engaged within it, yet create a high-certainty execution context. This ability to turn uncertainty into certainty is the key to … entrepreneurial effectiveness… Accurate predictions are not a sufficient for success. You need to ask the right questions. This requires leaving no stone unturned while you push forward on a singular mission to achieve your product vision.

The Superforecasting Playbook for Product Development

: offers the , a tool to help Data Scientists automate their weekly submission pipeline for the Numerai tournament. This CLI configures a in and automatically deploys your model to it. tl;dr Numerai numerai-cli Numerai Prediction Node Amazon Web Services (AWS) . 🚀 This guide describes how I set up my own weekly submission pipeline from scratch, using for free Microsoft Azure and python 🤖 Just give me the code: https://github.com/papaemman/azure-functions-with-python What is Numerai? is a quant hedge founded in San Francisco, CA, in 2015, built on thousands of crowdsourced machine learning models. Every week, Numerai gives away data so that Data Scientists worldwide have free, hedge-fund quality data to develop their Machine Learning (ML) / Deep Learning (DL) models — data they would otherwise not have access to. Numerai At the beginning of each round, competitors can stake (a cryptocurrency issued by Numerai) on the predictions they submit if they want to get paid (or "burnt") based on the predictive accuracy of their models. Staking on a model is fundamentally a type of "bet" competitors take towards the performance of their own model's predictions. NMR tokens Numerai Compute CLI Competing in Numerai is an ongoing, never-ending challenge because competitors need to download the new data every week and submit their predictions based on them. is a framework aiming to help participants automate their weekly submission workflow. Using the , participants can "provision" their cloud infrastructure in and deploy their pre-trained models as a Prediction Node that can Numerai Compute numerai-cli Amazon Web Services (AWS) be triggered by Numerai every weekend download new tournament data run the ML models and upload predictions to Numerai The configures a Numerai Prediction Node in Amazon Web Services (AWS). This solution is architected to cost less than on average, but actual costs may vary. numerai-cli $5/month You need 4 things to use Numerai CLI: , , , and . Docker Python Numerai API Keys AWS API Keys Why not just use Numerai Compute? 1. Cost Last week was tough for the crypto market. carried away the whole crypto market. Currently (May 2022), the NMR token has a price of around . Therefore, saving per month and investing them in buying more NMR to stake in your models is an appealing idea. The massive crash of LUNA 13$ 5$ 2. Learn something new When building something from scratch, you have the opportunity to understand how it works under the hood. Last year, I deep-dived into Microsoft Azure cloud technologies, and I wanted to test if my skills were enough to build a real-world solution. 4. Don't have an AWS account Many professionals prefer to work with other Cloud providers than AWS, such as or Google Cloud Platform. Setting up a new AWS account, adding billing information, and finding your way into the AWS console might seem like a lot of work. *, this guide is for you.* Microsoft Azure If you prefer to use Azure Goal Automate my weekly submission pipeline, for the Numerai Tournament, using Azure and python, with zero cost. Azure offers many services adequate to solve this task, but I decided to use Azure Functions after some research. is a cloud service available on-demand that provides all the continually updated infrastructure and resources needed to run your applications. You focus on the pieces of code that matter most to you, and Functions handles the rest. Functions provides serverless compute for Azure. Azure Functions Going for a is a great way to because you don't need to pay for the infrastructure you don't use. And if you already have the code you want to run, setting up an Azure function is . Also, Azure offers a which is more than enough for this project's scope. serverless architecture save money and time easy free tier, Thus my goal was to create an Azure Function that will: ⏲️ be triggered every Sunday 🔋interact with the Numerai API to download numerai_live_data and upload my predictions 💾 interact with Azure Storage to read and store and data pre-trained ML/DL models numerai_live_data numerai_predictions 📥 send a fun email notification after the submission 😊 allow me to enjoy my weekend MS Azure Architecture Design Technical Implementation Prerequisites Azure Account and VSCode Azure Extension Python 3.8 Azure Functions Core Tools Azure Storage Explorer Numerai API Keys Trained ML/DL models and inference python code Setup 1. Setup all prerequisites Download and install the required software Create your accounts in Azure and Numerai 2. Open VSCode and create a new local Azure function project While you don't have any workspace opened in VSCode, go to the extension, find and select _Create New Project…- S_elect , , and a to create a virtual environment- Select the template, give a name to the function, and set up the timer to run every [* 0 0 * * SUN]. You don't need to set up the function to run in short time intervals. Azure functions support " . Azure FUNCTIONS Directory Language (Python) python interpreter Time Trigger Sunday Execute Function now…" to trigger it manually within VSCode for testing purposes 3. Then, the VSCode will create a with a time-trigger function sample code. automatically Local project ## Azure function project codebase structure
.
├── .venv                     # Python Virtual environment
├── .vscode                   # Configuration options for VSCode
├── host.json                 # Configuration options that affect all functions in a function app instance
├── local.settings.json       # Maintains settings used when running functions locally.These settings aren't used when running in Azure
|
├── AzureFunction_1
│ ├── function.json           # Azure Function Settings
│ ├── __init__.py             # Python Code
│ └── readme.md               # Documentation
|
├── AzureFunction_2
│ ├── function.json           
│ ├── __init__.py             
│ └── readme.md               
|
└── requirements.txt          # Package dependencies (Open VSCode and Press F5 or 4. Run the Function locally Run →Start Debugging) To debug, you must select a storage account for internal use by the Azure Functions runtime: Select a , create a , create a and select a for resources. Keep in mind that this is a . subscription new storage account new resource group, location temporary resource group Go to extension, find and go to . Find the function, right-click, and select Azure FUNCTIONS Local Project Execute Function now… Examine functions' logs Stop the function (ctrl+c) Delete the resource group Important note ⚠️ ”, update the extensionBundle “version”: “[2.*, 4.0.0)” If you get an error “connect ECONNREFUSED 127.0.0.1:9091 to version , in host.json file. 5. Publish the project to Azure Go to extension_,_ find and select Azure FUNCTIONS, Deploy to Function App… Select a and subscription create a new function App Select a and for the new resource runtime stack location This will create a new resource group with an App Service Plan, a Function App, a Storage Account, and an Application Insights service. 6. Run the Function in Azure Go to extension, find , open the related and find the function app. Azure FUNCTIONS subscription Open the function app, find the specific function and select This will run the function to the Azure environment. Execute Function now…. You can see the function logs in the Live metrics tab using the Application Insights resource. 7. Download the function app settings Open VSCode and download remote settings in the Local environment for the function App ( and select Azure Functions: Download Remote Settings) F1 This will update the file with info about App service Plan, Function App, Storage Account, and Application Insights. local.settings.json 8. Setup the Azure Storage account. Open the connect with your Azure account and find the related storage account inside the newly created resource group. Azure Storage Explorer, Create a new Blob container called and upload all your pre-trained ML models there. production-models Create a new Blob container called to store numerai_live_data for every round. live-data Create a new Blob container called to store numerai_predictions for every round. predictions-files 9. Purchase the Twilio SendGrid SaaS offering from Microsoft Azure Marketplace To send a notification email every time the function triggered and submitted predictions, I decided to use the Twilio SendGrid's Email API Purchase the Twilio SendGrid offer (free tier) from Microsoft Azure Marketplace and configure your account on their platform. Get API keys from Twilio SendGrid 10. Configure Function App Having the and hardcoded in the source code is a terrible practice. Azure Functions offers a dedicated Function's App configuration to store environmental variables and secrets. Numerai API Keys Twillio Seng Grid API keys Open the , find the and go to . The complete set of Environment Variables is stored in the Application settings by default. This helps developers define input/output bindings with other resources in Azure. Azure portal Function App service Configuration Create 2 new application settings and store Numeai API Keys and SendGridKeys NumeaiAPIKeys →secret_key=***;public_id=***” SendGridKeyss →*** Open VSCode and re-download remote settings to the local environment ( and ) to get the new Application Settings into the file. F1 Azure Functions: Download Remote Settings local.settings.json 11. Write the Python code Now it's time to write (or copy-paste) the python code responsible for reading the numerai live data & pre-trained ML/DL models and submitting predictions using Numerai API. The input and output configurations (configurations and paths to load and store files) are the only thing you have to modify in your previous python code (running locally) to run it as an Azure Function. Instead of local paths, you should use the newly created Storage Account. For this, you need to use the . Azure Storage Python SDK Go to the file and replace your python code. .py init 12. Run the Function locally and verify that it is working Run the locally ( F5 or Run → Start Debugging), go to A_zure extension,_ find the Function in , and select ` Function Local Project ExecutFunctionon now… Check Function's logs and the connection with the Azure Storage Account Stop the Function (ctrl+c) 13. Redeploy and verify the updated App Go to extension, find , and select Azure FUNCTIONS Deploy to function App… After the deployment completes, find the Function under the respective subscription in VSCode, right-click to function, and select Execute Function now… Check Function's logs in the Metrics tab for this Function. Conclusions Last Sunday, I woke up to this fun email from myself, letting me know that everything went as planned. The Azure Function was triggered on Sunday at 00:00:00 UTC, the Numerai submissions were successful, and I was able to enjoy my day! 🌞 Email Notification for successful submission PS. Check the awesome quote at the end. I added a random quote generator inside the function creating the email notification just to spark things up ✨ Get in touch I hope you liked this guide. If you have any questions or just want to introduce yourself, don't hesitate to reach out! Panagiotis Papaemmanouil 📧 papaemman.pan@gmail.com 💬 https://www.linkedin.com/in/panagiotis-papaemmanouil/ 🌐 https://papaemman.com/ 🔥 Check my under the name Numerai Models PAPAEMMAN https://numer.ai/papaemman1 Also Published here

Automate Submissions for the Numerai Tournament Using Azure Functions and Python

If you believe that action and practice are better than theory and have little to no experience in the field of and quantitative analysis (but a healthy interest) , this might be the post for you, for the rest it is a look into a probable future where one more thing could become decentralized and open sourced: . machine learning Wall Street There are 2 main ideas at play here, so I’ll try giving you a birds eye view of both : Within the world of finance, hedge funds are a group of investors that plunk money into a basket of financial instruments and investments with hopes of having a return, a $3 Trillion Dollar competitive industry. Within this industry is one of the first experiments in open sourcing at some level the management and decision of buying and selling investments. Numerai: Numerai Quantitative analysis in finance employs statistical & mathematical models to predict and evaluate the price and value of financial instruments, in a nutshell various numbers representing some feature of the instrument go in, and a prediction or score comes out, said prediction or valuation then informs the decision making on the funds capital (buying,selling, holding etc) Machine learning which is everywhere these days is but the latest tool in making sense of the numbers; to a researcher like me or someone more interested in different A.I. problems like pattern recognition this type of analysis is just a new, fun and probably profitable application. Quantitavie Analysis & Machine Learning: Numerai bridges both worlds and adds a crowdfunded twist by providing the raw data and asking you and me to provide a better estimate, these predictions are then used to control capital and the top performing ones get paid real money for their contribution, additionally there are incentives in the form of staking your own funds ( in the form of numerais own cryptocurrency) which in essence is a reward and punishment system for keeping your predictions novel and useful. This is a complex but interesting part which I will sidestep for now to focus on getting started. For the money, for the glory, and for the fun , but mostly for the money. In my experience it will take a novice data scientist, coder, or hobbyist a few weeks to get off the ground, at the end of this very short period you will have practical knowledge on applying various statistical and A.I. concepts to a real problem on a platform that you can then expand to your own interests, the competitive aspect and financial incentives can help you stay focused and motivated while learning although I should mention that making a dollar might take months. If you are not planning on participating but came for the whole open source wall street thing you can skip the following 2 sections. Hands on: This is but one path ( the one I took ) yours might be different depending on your starting point. Sign up to Numerai, read the documentation, get thoroughly intimidated by the documentation, the forums, the slack channel and the sinking feeling that even though you thought you were a smart cookie, this will require learning quite a few new things, take a deep breadth. Download the data set, it consists of multiple files which you will use, here’s what the latest one looks like: These files might make perfect sense if you’ve done some , if not I’ll try giving you a quick overview/introduction: data science Numerai is kind enough to provide 2 fully functioning and working examples, these are perfectly good springboards and working examples of a base implementation in R and Python, 2 very popular languages for big data analysis and machine learning, I went with Python since I had some experience with it, although I foolishly tried making a neural net on excel at first, my computer is still recovering. example_model.py & example_model.r : There are a few things to this file, so let’s start by looking at what’s inside: numerai_tournament_data.csv : Importantly there are various types of elements or categories ( id’s) divided into validation,live and test types, and each one has features, and some date information in the form of eras; the target column is what your model will predict in the form of a binary number (1 or 0). Numerai obfuscates the data, so even though we are assuming these are financial instruments (could be volume, open,close price, etc) there is nothing explicitly apparent, feature 11 on id 12546 could represent stock price at close date last week (era2) or something else, a dataset representing cancer rates or pixels in a dog recognition problem would look the same ! numerai_training_data.csv: An additional dataset, the training set is also provided, if you are familiar with it will be obvious what this dataset represents and how it is used. In a nutshell, you are given a problems definition and it’s solution, ( ), you a model to it and then generalize the solution to new data ( ) and make your predictions. It is the equivalent of learning to ride a bicycle with training wheels ( here you are the model ) and then trying your skill (predicting) by riding on a bicycle without them, if you fall you have a bad model. cross validation the training set fit the blank test and live rows on the tournament dataset This is an example of what you will be submitting, a column of id’s ( the same id’s from numerai_tournament_data.csv) and a probabilistic prediction that the id will be of class 1, so while the target of your model is a binary number, it needs to output probabilities on said target. example_predictions.csv : Once you have these files, you now need to set up your environment for making, predicting, keeping track of and submitting predictions. In my case it involved installing python plus setting up my editor ( I use Atom with script for running python) and creating a few spreadsheets to test,tweak and keep track of predictions, this is time consuming and will probably take a few weeks if not months to streamline. As mentioned, the example models provided by numerai are an excellent way to get started making predictions and working with models, the code does everything for you (read files, divide data, define a model, train,predict, write to file ) provided you have the right libraries installed, pandas and numpy for dealing with the data and sklearn for the statistical models for instance. Models : Numerai has a weekly tournament and after each one the clock resets and a new dataset is provided, you can have 3 models submitting predictions and are limited to uploading 25 per day (it’s usually plenty once you learn how to calculate logloss and other metrics on your own). They also provide an API if you choose to automate the download/upload. The Tournament : Once you have successfully submitted a model, you will be in the leaderboard and your predictions will be scored across various dimensions : At this point the algorithms for originality,concordance and consistency are not open source, but they are described as follows ( ) you also need to meet a specific treshold to qualify for the tournament and control capital Logloss across era, explanation forthcoming. ( ) Consistency: 75% an up Somehow controversial since it might favor early submitters; a check to see if your predictions/model were already submitted. ( a pass/ fail flag) Originality : A check to see if one model was applied to all data_types. ( a pass/fail flag) Concordance : Once your model is prequalified by these metrics, you are then scored by your logloss against live data in order to actually win money, a process that takes 4 weeks, the scoreboard is then not a reflection of the actual place your model will place in the tournament. Numerai is not telling the specifics, but they hint at a meta model based on the top predictions, those closests in logloss to their own model and their live data, the time period also plays into favoring models and predictions over a longer time span, since logloss is the main ranking metric, it’s important to know about it. is a graded statistical measure of confidence over your predictions, in real terms something like 0.50 would represent a random guess and 0.00 a perfectly confident prediction, it is graded because the more confident you are about a certain outcome ( I will not fall of the bicycle) the more it penalizes you if in fact you fall of the bicycle. In the case Numerai it could be understood as I am certain to this degree about my predictions. Logloss : ( or logarithmic loss ) And that’s it, once you have your setup ready you are welcome to try any of the multiple statistical models that are open source, and any technique and combination of parameters for analyzing this data, a good place to start is the sklearn library and all their models. I should also mention that ranking within the 100 or so models that are paid is difficult and requires a lot of experimenting, applying advanced techniques and in general getting to know the dataset, so using out of the box models will probably not get you very far. : Folks interested on the whole open source Wall Street thing continue reading here Beyond a great way to get into applied machine learning and quantitative analysis, Numerai might become something bigger and arguably better for the world of finance, it is early days and it is on the experimental side like many startups, but the upside potential is there. The hedge fund and banking industry have traditionally been elite,closed and secretive places with a bad reputation, in the same way that Hollywood and the TV networks were once the only source for movies and shows, Youtube has provided a new avenue for both consumers and content creators. A distributed and open source wall street would allow diverse talent and ideas to contribute, a self taught analyst in a developing country that does not like wearing a suit and the heavy handed bro atmosphere of wall street could make a living and contribute to a hedge fund. A researcher in academia might not want to abandon his full time job or research, but might be able to contribute part time. The consumers of the hedge fund industry as a whole would also benefit if the experiment proves successful, this bet hinges on the wisdom of the collective quant crowd vs the existing industry and the performance of these new types of funds. I am personally somehow optimistic, but I am also aware that consistent positive performance in the hedge fund industry (and other investments) usually favors an index approach vs stock picking and active management, so in the end a quant approach, even an open sourced one might not be all that profitable. Finally, I believe the hedge fund industry will probably have nothing to fear,but will coexist peacefully and even contribute if the experiment resolves favorably, established banks might even one day foster this type of funds, competition wise, I also think there is room for everyone since there is plenty of untapped investment capital facilitated by the new ecosystems (cryptocurrencies,globalization, the web) which numerai an other similar ventures might be better positioned to capture; if there is a point of concern for the hedge fund industry, is that of cost, the current fee structure ( 2 percent on assets and 20% on profits ) is pricey for 2 reasons, the cost of having an infrastructure (brokers, analysts, researchers, back office, etc) and the walled garden of finance that limits participation, it is in this space that a desentralized open source hedge fund can lower costs and compete, so as mentioned we could very well see a new more competitive type of hedge fund appear and the existing ones would need to adapt. 2 and 20 Parting Words I left a few things out for lack of space and because I wanted to focus this short piece as a both an alternative way of onboarding, which I struggled with and to discuss albeit briefly what this type of fund could herald. You can read about staking via this , and keep up with numerai at their . whitepaper blog I should also mention and other writings on , and here on medium as a way to start and further your learning (as well as mine). kaggle numerai machine learning quantitative analysis Small Update: (Oct 1st 2017)The dataset has been changed, there is now roughly twice the features and 10–20 K more rows, this means that if you are on basic hardware ( I am on a i3 4GB RAM HD windows machine) you will most likely experience memory errors on some models, the solution is to subsample. Also, I got my first NMR by ranking within the first 100 ! Medium sized Update: (Dec 25 2017 ) Numerai has now released the source for scoring predictions and streamlined the API for automatic submissions along with some other minor changes and a vision statement & road map, read more here : Most importantly, payouts have now changed in nature (more NMR, less usd). I personally have been doing great ranking in the top 50 – 60 more or less consistently. Numerai’s Master Plan. Early 2018 Update — Breaking changes: Payouts for the tournament have changed once more, the only way to win both NMR and USD . This means that in order to compete you need to have NMR to stake , and in order to have NMR you have to buy them or have been part of the original distribution, it is early to know how this will affect the project, but in my eyes the project has changed in nature, moving the bar higher and making it harder for newcomers, time will tell if this breaks the project. is through staking Mid 2018 Update — More changes: There are now 5 different tournaments and the payout structure has gone through a number of complex rule changes, consult the Numerai page for the current ones, there is also a new reputation system that rewards you with 0.1 NMR every time you beat benchmark, ( there was also a sweet reputation bonus for previously submitted performant models ); so now you have a path to stake NMR without buying them, I think it’s perhaps too small an amount, but understand there are issues in giving out more (like bots). I’ve been consistently ranking in the top 20 but earnings are now tied to how much you stake, your confidence level and that of other players,coupled with the current cryptocurrency slump, my earnings have been low ; there have also been problems with bots taking advantage of airdrops and players exploiting rules, these have all been progressively addressed by Numerai resulting in added complexity and rules. Disclaimer : I just took a short term contract position with Numerai (August 2018-Oct 2018) I will be helping them with Front End,UX and enything else I can. I won’t be competing during the duration… April 2019 Update : There’s been quite a few new things in the world of Numerai since I left and went back to being a competitor, there’s new data, new tournaments ( still only 5 though) and a new measuring metric (AUC) instead of logloss, it has also become much more competitive ( ), mostly pay to play and payouts per participant have gone down, ( although general prize pools are bigger now). I rank top 100-200 now Thanks ! Keno About the Author : Born Eugenio Noyola Leon (Keno) I am a Designer,Web Developer/programmer, Artist and Inventor, currently living in Mexico City, you can find me at www.k3no.com

Numerai walkthrough: Quantitative Analysis & Machine learning for fun and profit.

and are two of the most transformative forces shaping the modern world. AI has the potential to revolutionize industries by automating processes, enhancing decision-making, and unlocking new insights from vast amounts of data. Artificial Intelligence (AI) blockchain technology Blockchain, on the other hand, offers a decentralized and secure framework for digital transactions, enabling trust and transparency in an increasingly interconnected world. The convergence of AI and blockchain has given rise to a new breed of cryptocurrencies known as AI crypto coins. These digital assets leverage the power of AI to enhance various aspects of the blockchain ecosystem, from improving the efficiency of decentralized networks to enabling more intelligent decision-making in financial markets. In this article, we will explore the concept of AI crypto coins, their underlying mechanisms, and their potential applications. I will try to explain this topic based on my experience of working  with crypto and AI projects. In this article, we will explore the benefits and risks associated with AI crypto coins, as well as how to buy them and what to consider before investing. This article should not be regarded as financial advice. Always remember to DYOR! Understanding AI Crypto Coins Unlike traditional cryptocurrencies, AI crypto coins use machine learning algorithms and natural language processing to analyze market patterns, make predictions, and execute automated trades. This allows them to adapt to changing market conditions in real-time, providing investors with a more agile and responsive investment option. , which are automated computer programs, are a key component of AI cryptocurrencies. They can be used for automated trading strategies, amplifying trading profits by applying these strategies to multiple cryptocurrencies on one or diverse platforms. Some of the most known AI trading bots include Defiquant, Pionex, GunBot, Kryll, Zignaly, Dash 2 Trade, ArbitrageScanner, and Cryptohopper. AI trading bots These bots collect big data on crypto information in real-time and analyze it to derive useful and actionable metrics, with the target of future crypto prices. They can pull information from various social sources, crypto markets, news sites, and other sources to provide valuable insights for investors. AI trading bots are beneficial for modern investors, as they can be configured to run automatically, which is particularly useful for copy trading and building a bot from scratch. They can also be used for social trading, paper trading, backtesting, and strategy design. Benefits of AI Tokens Improved Decision-Making One of the key benefits of AI crypto coins is their ability to improve decision-making. By analyzing large volumes of data, AI algorithms can spot patterns and trends that human analysts might miss, helping traders make more informed decisions. This can lead to better risk management and improved returns on investment. For example, AI algorithms can analyze market trends, social media sentiment, and news articles to predict price movements. This can help traders make more accurate predictions and reduce the risk of losses. Enhanced Network Security AI crypto coins can also enhance network security. For example, some AI cryptocurrencies use facial recognition technology to verify transactions and prevent fraud. This can provide an additional layer of security, making it more difficult for hackers to gain access to sensitive information. AI algorithms can also detect and prevent cyber attacks in real-time, providing an added level of protection for investors. Automation of Functions Another advantage of AI crypto coins is their ability to automate numerous functions in the cryptocurrency market. By using smart contracts, AI algorithms can execute trades automatically, speeding up transactions and saving time. This can also reduce the risk of human error, which can be a significant factor in the fast-paced world of cryptocurrency trading.For example, AI algorithms can automatically execute trades based on pre-defined criteria, such as price movements or market conditions. This can help traders take advantage of opportunities more quickly and efficiently. Notable AI Crypto Tokens 1. SingularityNET (AGI) SingularityNET is a decentralized AI marketplace that enables users to access a wide range of AI services through the use of AGI tokens. By leveraging a decentralized network of AI algorithms, SingularityNET aims to democratize access to AI technology and foster innovation within the crypto AI landscape. As of March 2023, SingularityNET has over 50 AI services available on its platform and has partnered with major organizations such as IBM and the United Nations. 2. Fetch.AI (FET) Fetch.AI is an AI-powered decentralized network that enables the creation of intelligent autonomous agents. These agents can perform various tasks, such as optimizing supply chain logistics, facilitating peer-to-peer trading, and providing personalized recommendations. FET tokens are used to access the Fetch.AI network and incentivize participation. Fetch.AI has raised over $15 million in funding and has partnerships with organizations such as Bosch and Datarella. 3. Ocean Protocol (OCEAN) Ocean Protocol is a decentralized data exchange platform that enables users to securely share and monetize data through the use of OCEAN tokens. By leveraging AI capabilities, Ocean Protocol aims to create a more equitable and transparent data economy, ultimately benefiting data providers, consumers, and developers alike. Ocean Protocol has raised over $20 million in funding and has partnerships with organizations such as Aviva and Roche. 4. Numerai (NMR) Numerai is a decentralized hedge fund that leverages AI algorithms to make investment decisions. Contributors to the Numerai platform can submit their AI models, which are then used to inform the fund's trading strategies. In exchange for their contributions, users are rewarded with NMR tokens. Numerai has over 40,000 data scientists contributing to its platform and has over $1 billion in assets under management. 5. Cortex (CTXC) Cortex is a decentralized AI platform that enables the creation of intelligent smart contracts. By leveraging AI capabilities, Cortex aims to improve the performance, scalability, and security of smart contracts, ultimately leading to the development of more sophisticated and intelligent dApps. CTXC tokens are used to access the Cortex network and incentivize participation. Cortex has raised over $10 million in funding and has partnerships with organizations such as Binance and Bitmain. Risks Associated with AI Crypto Coins However, there are also risks associated with AI cryptocurrencies. The lack of regulatory oversight in the market can create an environment where scams and fraud are more likely to occur. AI cryptocurrencies are also subject to high levels of volatility, which can result in significant losses for investors. Technical issues, such as errors in the smart contract code, can also lead to loss of funds or security breaches. Ethical concerns, such as the potential for bias in decision-making or the impact on employment in the financial industry, should also be considered. How to Buy AI Crypto Coins To buy AI crypto coins, investors need to find a cryptocurrency exchange that supports the specific AI crypto coin they want to purchase. After creating an account and funding it with a payment method such as bank transfer, credit card, or cryptocurrency transfer, investors can use the exchange to buy the AI crypto coin of their choice. Once purchased, it is recommended to transfer the coin to a secure cryptocurrency wallet to ensure the safety of the investment. Ultimately, AI crypto coins have the potential to become a valuable addition to any investment portfolio. One key area where AI crypto tokens could make a significant impact is in the realm of decentralized finance (DeFi). It could lead to the creation of new investment opportunities and the democratization of financial services, making them more accessible to a broader audience. If you’re interested in the most innovative AI projects and research, you’re welcome to subscribe to my newsletter ‘ ’. It’s free! AI Hunters P.S. Check out some of my previous articles at HackerNoon: Google Introduces Gemma: Revolutionary Access to Pre-Trained AI Models Sora's AI Innovation Set to Disrupt Video Content Creation Breaking Down the Next Big Thing in NFTs: ERC-404 Spotlight Gemini 1.5 Unleashes Unprecedented Context for AI Applications The AI Revolution in Journalism: A New Era of Enhanced Reporting DeFi Community Building: A Step-by-Step Guide for Crypto Startups

AI Crypto Coins: When the Power of AI Meets the Blockchain

Authors: (1) Gopal Yadav, Department of Physics, Indian Institute of Technology & Chennai Mathematical Institute. Table of Links Abstract Acknowledgment PART I Chapter 1: Introduction Chapter 2: SU(3) LECs from Type IIA String Theory Chapter 3: Deconfinement Phase Transition in Thermal QCD-Like Theories at Intermediate Coupling in the Absence and Presence of Rotation Chapter 4: Conclusion and Future Outlook PART II Chapter 5: Introduction Chapter 6: Page Curves of Reissner-Nordström Black Hole in HD Gravity Chapter 7: Entanglement Entropy and Page Curve from the M-Theory Dual of Thermal QCD Above Tc at Intermediate Coupling Chapter 8: Black Hole Islands in Multi-Event Horizon Space-Times Chapter 9: Multiverse in Karch-Randall Braneworld Chapter 10: Conclusion and Future outlook APPENDIX A APPENDIX B APPENDIX C Bibliography CHAPTER 7 - ENTANGLEMENT ENTROPY AND PAGE CURVE FROM THE M-THEORY DUAL OF THERMAL QCD ABOVE T AT INTERMEDIATE COUPLING c 7.1 Introduction and Motivation In doubly holographic approaches, as described in 5.3.2 of chapter 5, a gravitational dual black hole is connected with an external CFT bath [202]. As an illustration, gravity in d-dimensions is connected with the external bath in d-dimensions, wherein d-dimensional exterior bath possesses a corresponding holographic dual in (d + 1)-dimensions. In such models, we take into consideration two variants of the previously mentioned configuration. We look at two distinct types of extremal surfaces in these models. The very first one corresponds to the Hartman-Maldacena-like surface [144], that begins from the position at which gravity interacts with with an external bath, i.e., at the defect, and traverses the black hole horizon to get to the defect of the thermofield double associated with doubly holographic setups, and the entanglement entropy involvement via the aforementioned surface possesses linear time dependency, leading to information paradox at later times. The other type of surface includes the island surface, that begins at the exterior bath and ends on the KarchRandall brane [142, 143]. The contribution of entanglement entropy coming from the island surface becomes time independent and takes over beyond the Page time. The Page curve is obtained through the combination of the entanglement entropy contributions obtained from both the extremal surfaces. As we discussed in 5.3.2, in some cases, the gravity becomes massive on the Karch-Randall branes, and one was unable to get the Page curve with massless gravity localized on the KarchRandall branes. In this chapter, we have looked at this major issue of doubly holographic setup from M-theory perspective in the presence of O(R4 ) terms in the supergravity action. We have a different situation from the literature that we have a non-conformal bath (QCD bath), and the doubly holographic setup is being constructed from a top-down approach[1] with the inclusion of higher derivative terms in the supergravity action. Interestingly, the issue of the massive graviton is absent in our setup, and one can get the Page curve with massless gravity on the end-of-the-world(ETW) brane. We cannot utilize the island approach for non-conformal backgrounds because currently there is no recognized Cardy-like formula for the non-conformal theories. As a result, we used the previously mentioned doubly holographic frameworks technique to arrive at the Page curve. In the context of our scenario, on the gravity dual side, there is a M-theory dual consisting of higher derivative terms, whereas on the gauge theory side, we find nonconformal thermal QCD-like theories at the intermediate coupling, the model is discussed in 1.4 of chapter 1. In the present chapter, we created a doubly holographic setup analogous to [7,203], where the researchers utilize a conformal theory on their gauge theory side. There are two types of potential surfaces to think about: Hartman-Maldacena-like and island surfaces. The Hartman-Maldacena-like surface has been accountable for the gradual linear temporal growth of Hawking radiation’s entanglement entropy. Above the Page time, the entanglement entropy component via the island surface takes over and this is not dependent upon time, so we extract the Page curve using the M-theory dual, which includes O(R4 ) corrections. As a remark, we used the formula of [127] to compute the entanglement entropy with the inclusion of higher derivative terms and when higher derivative terms are absent then we used the Ryu-Takayanagi formula to compute the entanglement entropy [107]. 7.2 Doubly Holographic Setup in M Theory From the work of [127], we could compute the entanglement entropy associated with holographic dual theories with the inclusion of higher derivative terms. We could express the analogue of the generalised entropy functional described in equation (5.17) using [127] for M-theory dual in the presence of O(R4) terms as follows: and inserting it into (7.20) yields: 7.3.1 At O(β) and we assume the constant is zero. 7.3.2 No Boundary Terms Generated at O(R4) The eleven-dimensional supergravity action, which includes O(R4 ) terms, is presented by the following: where, As a result, we are able to identify that a usual boundary term including covariant derivatives of metric fluctuations that would have to be cancelled out by a suitable boundary term (using Stokes theorem) is: The (dual to the) unit normal vector to provided by: W Hence, Utilizing, 7.4 Page Curve without Higher Derivative Terms Here, we obtained the Page curve using the doubly holographic setup constructed in 7.2. In this part, we do not consider the ( ) terms in the supergravity action. Therefore, we used the Ryu-Takayanagi formula to compute the entanglement entropies of Hartman-Maldacena-like and Island surfaces in 7.4.1 and 7.4.2. The Page curve has been obtained in 7.4.3. In the absence of ( ) terms, the = 1, = 11 supergravity action has been provided as follows: O R4 O R4 N D 7.4.1 Entanglement Entropy Contribution from Hartman-Maldacena-like Surface To calculate the time-dependent entanglement entropy associated with a Hartman-Maldacena-like surface, we first look at the induced metric on a constant x 1 slice, which was calculated utilizing (7.20) as follows: The area density functional for a Hartman-Maldacena-like surface could be calculated using (7.23) as follows: implying Using (7.27) and (7.30), we obtained: Now we identify time via the integral: As a result, the Hartman-Maldacena-like surface’s entanglement entropy is: 7.4.1.1 Hartman-Maldacena-like Surface Analytics/Numerics After integrating all of the angular coordinates and consequently including a (2π) 4 emerging from integration w.r.t. ϕ1,2, ψ, x10, the area corresponding to the Hartman-Maldacena-like surface is obtained as: As a result, the entanglement entropy related with a Hartman-Maldacena-like surface grows linearly with time. 7.4.2 Entanglement Entropy Contribution from Island Surface We take a constant t slice to calculate the entanglement entropy for an island surface. As a result, utilizing equation (7.20), we expressed the induced metric associated with the island surface as follows: where C is the constant. We simplified the expression (7.43) utilizing equation (7.41) as given below: When we solved the preceding equation for x˙(r), we were provided with: Constant could be calculated using the equations (7.45) and (7.46), as shown below: C Taking the previously stated expression of C, the equation (7.44) reduces to the following structure: Now, the area density functional associated with the island surface has been simplified using the equations (7.41) and (7.48). Hence, the entanglement entropy associated with the island surface is obtained as follows: Since, there is no time dependence in (7.50), and hence entanglement entropy of the Hawking radiation for the island surface is constant. 7.4.2.1 Island Surface Analytics When (7.49) is evaluated, the entanglement entropy is as follows: implying 7.4.3 Page curve at LO from Areas of Hartman-Maldacena-like and Island Surfaces 7.5 Page Curve with Higher Derivative Terms We obtained the Page curve corresponding to a neutral black hole in with the inclusion of higher derivative terms that are quartic in the Riemann curvature tensor in this section. This section has been broken into four subsections. We calculated the entanglement entropy associated with the HM-like surface in subsection 7.5.1, discussed the “Swiss-Cheese” structure of the identical surface in subsection 7.5.2, computed the entanglement entropy of island surface in 7.5.3, and at last obtained the Page curve corresponding to an eternal black hole in subsection 7.5.4 utilizing the outcomes that were obtained from previous subsections. We conducted the same analysis as in 7.4. [127] was used to compute the entanglement entropy associated with Hartman-Maldacena-like surface and island surface in with the inclusion of higher derivative terms. In generic higher derivative gravity theories, we are able to calculate the holographic entanglement entropy (5.8) as follows: • When the holographic dual consists of a ( + 1) dimensional gravitational background, then with the help of the embedding function, compute the induced metric associated with the co-dimension two surface. d • Using the above-mentioned induced metric, compute (5.8). This yields the holographic entanglement entropy as an expression of the embedding function and its derivatives. • Find the solution to the embedding function’s equation of motion. • In higher derivative gravity theories, putting the solution found in the previous step inside the action yields the holographic entanglement entropy. To determine the second part in the formula (5.8), we must first compute the four types of derivatives for the Hartman-Maldacena-like and island surfaces, which are listed below: The numerator and denominator coefficients in the preceding equation are x = xR for the Hartman-Maldacena-like surface and x = x(r) for the island surface. All four types of variations occurring in equation (7.54) for the Hartman-Maldacena-like surface in appendix C.5.1 and island surface in appendix C.5.2 have been computed and listed. 7.5.1 Entanglement Entropy Contribution from Hartman-Maldacena-like Surface Hartman-Maldacena-like surface in M-theory dual corresponds to a co-dimension two surface situated at x 1 = xR. Utilizing equation (5.8), we are able to write the mathematical expression that describes the entanglement entropy associated with a Hartman-Maldacena-like surface as follows: where g denotes determinant of the induced metric defined in (7.23) for the co-dimension two surface. For the metric (7.23), the entanglement entropy associated with the Hartman-Maldacena-like surface using (7.55) and appendix C.5.1 after the angular integrations is obtained as: where The equation (7.58) has the following solution: substitution of (7.59) in (C.20) yields: Keeping the term up to the leading order in N, we could write the embedding function t(r) as follows: The entanglement entropy of the Hawking radiation corresponding to the Hartman-Maldacenalike surface is computed as given below: utilizing (C.13) and (7.59), (7.66) is expressed as: Similarly, 7.5.3 Entanglement Entropy Contribution from Island Surface The island surface is a co-dimension two surface that exists at a continuous time slice, much like the Hartman-Maldacena-like surface. Therefore, we can formulate the formula for the holographic entanglement entropy as follows: Hence, we found that: and, Thus, the island surface’s equation of motion associated with embedding x1(r) is as follows: (7.89)’s solution is provided as: implying, The above equation has the following solution: The black hole entropy for the metric (7.19) is obtained as: Therefore, the entanglement entropy for the island surface after substitution of β simplified as: From Fig. 7.5 and utilizing (7.101), it is obvious that (7.97) ∼ (7.98) means that a lower constraint will be placed on rh, the non-extremality parameter in the M-theory dual of large-N thermal QCD. 7.5.4 Page Curve and An Exponential(-in-N) Hierarchy of Entanglement Entropies up to O(R4) Before/After the Page Time At the Page time, the entanglement entropies for the Hartman-Maldacena-like surface and the Island Surface are equal. This results in: Page time: Utilizing (7.99), (7.104) implies, Similarly, for the island surface O(β) contributions to the entanglement entropy is obtained as: Utilizing (7.99), (7.107) results in: From the equations (7.71), (7.100), (7.105) and (7.108), we obtained the following hierarchy: 7.6 Massless Graviton - the Physical reason for Exponentially Suppressed Entanglement Entropies Similar to [209], we can write the eleven-dimensional metric (7.19) as given below: We found that: the solution of above equation is: Since (7.121) is not well defined in the UV when m ̸= 0. Therefore, we need to consider m = 0 for which the graviton wave function is: Figure 7.8 plots the aforementioned potential for massless graviton (m = 0); this potential is “volcano”-like, with the massless graviton localized at the horizon on the ETW “brane”. This is analogous to [142], in which gravity can be localized on the end-of-the-world (ETW) brane with non-zero brane tension owing to the emergence of a “crater” in the “volcano” potential in the Schrödinger-like equation of motion of the graviton wave function. 7.7 Summary In this chapter, we constructed the doubly holographic setup from a top-down approach as discussed in 7.2. We have taken the external bath as thermal QCD in three dimension whose holographic dual is M-theory inclusive of O(R4) corrections [1], see chapter 1 for more detail of the same. The intermediate description of the doubly holographic setup constructed by us couples the black hole on the ETW-“brane” at x = 0 to thermal QCD bath via transparent boundary condition at the defect. We have addressed the effect of higher derivative terms in the context of top-down construction of double holography for the first time. First we obtained the Page curve without inclusion of O(R4) terms in the eleven-dimensional supergavity action in 7.4. To achieve the same, we computed the entanglement entropies associated with the Hartman-Maldacena-like (HM-like) and island surfaces and obtained the Page curve with the help of these entropies because the HM-like surface has Based on the viability of the Islands scenario, the M-theory dual of large-N thermal QCD at high temperature obtained in [15], [1] (based on [14]) produces the previously mentioned Page curve. The above-mentioned top-down M-theory dual, on the other hand, provides a number of new conceptual information: • Due to the lack of details of precise structures for boundary terms on the ETW-“brane” along with the presence of higher derivative terms, there exists relatively few works (e.g., [147]) in which the authors look into the doubly holographic setup in higher derivative theory of gravity. Remarkably we showed in 7.3.2 that the presence of the O(R4) terms generates no boundary terms. • As for as we know, our setup/(above-mentioned) M-theory dual appears either the sole one in existence (from M-theory) or one among the comparatively few top-down approaches that generate(s) the Page curve for massless graviton. • In our top-down M-theory dual, we find that ETW-“brane” to be a fluxed hypersurface W that is a warped product of an asymptotic AdS4 and a six-fold M6 where M6 is a warped product of the M-theory circle and a non-Einstenian generalization of T 1,1 ; the hypersurface W, can also be thought of as an effective ETW-“brane” corresponding to fluxed intersecting M5-brane wrapping a homologous sum of S 3×[0, 1] and S 2×S 2 in a warped product of R 2 and an SU(4)/Spin(7)-structure eight-fold. The ETW-“brane”, W, then has non-zero “tension” and a massless graviton localized near the horizon by a “volcano”-like potential. • Unlike largely other works in the literature, which compute the Page curve with a CFT bath, the external bath in our model is a non-CFT bath (thermal QCD). • Entanglement entropy contribution generated by a Hartman-Maldacena (HM)-like surface, which has been causing the growth of the Einstein-Rosen bridge in time, shows a Swiss-Cheese structure in the Large-N scenario (7.5.2) as discussed earlier. • By the presence of O(R4 ) terms in the action, our previously discussed M-theory dual produces a hierarchy in the entanglement entropies of the HM-like and Island surface (IS) with respect to a large-N exponential suppression factor, resulting physically via the presence of massless graviton mode on an ETW-brane. This suppression also suggests the fact that the addition of higher derivative terms - O(R4) in particular - has no effect on the Page curve. • To regulate the IR- and large-N enhancement in the IS entanglement entropy per unit BH entropy, a relationship involving the Planckian length and the non-extremality parameter (the horizon radius) has been shown to occur. • The positivity of the Page time, calculated in (refPage-time), has been showed to set an upper constraint on the non-extremality parameter, the black-hole horizon radius rh. This paper is under CC 4.0 license. available on arxiv

Entanglement Entropy and Page Curve From the M-Theory Dual of Thermal Qcd at Intermediate Coupling

This paper is available on arxiv under CC BY-NC-SA 4.0 DEED license. Authors: (1) Ke Hong, Tsinghua University & Infinigence-AI; (2) Guohao Dai, Shanghai Jiao Tong University & Infinigence-AI; (3) Jiaming Xu, Shanghai Jiao Tong University & Infinigence-AI; (4) Qiuli Mao, Tsinghua University & Infinigence-AI; (5) Xiuhong Li, Peking University; (6) Jun Liu, Shanghai Jiao Tong University & Infinigence-AI; (7) Kangdi Chen, Infinigence-AI; (8) Yuhan Dong, Tsinghua University; (9) Yu Wang, Tsinghua University. Table of Links Abstract & Introduction Backgrounds Asynchronized Softmax with Unified Maximum Value Flat GEMM Optimization with Double Buffering Heuristic Dataflow with Hardware Resource Adaption Evaluation Related Works Conclusion & References 3 Asynchronized Softmax with Unified Maximum Value The partial softmax operation requires synchronization among different partial vectors, leading to ∼20% overheads of the attention operation. As is shown in Figure 2, the synchronization is required after the maximum value of the partial vector is calculated. The maximum value is used to update previous partial softmax (i.e., recompute previous attention) results. Thus, to reduce synchronization overheads, Motivation. the key problem to be solved is how to compute each partial softmax result without requiring results from other partial softmax computation. The reason that synchronization is required lies in that the maximum value of each partial vector is different. The maximum value is used to avoid overflow of the exponent operation (f(x) in Figure 4(a)), and exponents are summed (l(x) in Figure 4(a)) as the denominator of the softmax operation. Such a non-linear operation on each partial maximum value makes the synchronization among each partial softmax computation unavoidable. Challenge. According to the formula of softmax computation, the maximum value is used as the scaling factor for both the numerator and the denominator (i.e., f(x) and l(x) in Figure 4(a)). Our key insight is, the scaling factor can be an arbitrary number rather than using the maximum value mathematically, shown in Equation (3). When we set ϕ = 0, it becomes the original softmax computation [30]. Analysis and Insights. However, the scaling factor cannot be an arbitrary number considering the overflowing of the exponent computation. For the case where xi ≫ ϕ, e xi−ϕ overflows and cannot be represented using a fix-width floating point number (e.g., float32 for exponent results in current LLM engines). For another case where xi ≪ ϕ, e xi−ϕ → 0, leading to precision loss. Thus, a proper scaling factor ϕ should be carefully selected to avoid the two cases above. Figure 5 shows the statistical distribution of xi (elements in the input vectors of softmax) in typical LLMs with different inputs [31]. Our key insight is, > 99.99% xi Specifically, for Llama2-7B, we have −16.8 < xi < 6.5 for > 99.99% xi . Because e b−a and e a−b can be represented by a float32 format, we can set ϕ = a in Equation (3). For OPT-6.7B, we do not apply the technique in this section because of the large range in Figure 5. are within a certain range. Based on the insights above, each partial softmax computation shares a unified maximum value, ϕ. After the softmax operation, an inner product operation is executed between the softmax result and a column of V (i.e., v). Assume that the input vector x can be divided into p partial vectors, x = [x (1), ..., x(p) ] (v = [v (1), ..., v(p) ] correspondingly), we have: Approach: Asynchronization. The inner accumulation in both the numerator and the denominator only take the partial vectors x (j) and v (j) as input, thus they can be processed asynchronously and individually. The outer accumulation is only processed after all partial vectors are processed. As we can see in Figure 4(c), each f(x (j) ) is calculated individually, and sof tmax(x) is calculated after all x (j) is calculated. Without loss of generality, we assume a < xi − ϕ < b for each xi to ensure precision and avoid overflow. Then, the partial softmax operation is processed individually. However, when xi − ϕ ≤ a or xi − ϕ ≥ b, the asynchronized partial softmax computation is terminated for the vector x where xi belongs to. The softmax is then recomputed using the synchronized partial softmax scheme (used in FlashAttention [18, 19] and FlashDecoding [13]) shown in Figure 4(b). Such a recomputation scheme avoids overflow while introducing negligible overheads based on the statistical data shown in Figure 5. Approach: Recomputation.

FlashDecoding++: Faster Large Language Model Inference on GPUs: Asynchronized Softmax with Unified

229 Stories To Learn About Data Analytics

253 Stories To Learn About Data Analysis

361 Stories To Learn About Big Data

Let's learn about via these 85 free stories. They are ordered by most time reading created on HackerNoon. Visit the to find the most read stories about any technology. Mathematics /Learn Repo I don't like math, but you might. The wonderous world of numbers awaits! 1. Implementing 3x+1 or Collatz Conjecture In Python 3x+1 or Collatz conjecture is a simple maths problem that can easily be implemented using a simple while loop in Python. 2. Why AI is a Fear-Driven Discipline People are scared of AI. According to Genpact research: 3. 69 Poem 69 was in his house 4. Java Program to Check for Right Angled Triangle A simple program in Java to check whether a triangle is right-angled or not. 5. Urpflanze JavaScript Library for Generative Art and Creative Coding Urpflanze is a javascript library to approach creative coding and generatve art 6. Formulas of Uniswap: A Deep Dive Find out the benefits of concentrated liquidity for multiple pooled assets in Uniswap. 7. You Can't Run from Math: Applications of Integration & Differentiation Learn all about Integration and differentiation with examples and understand their role in calculus. 8. A Guide to Two's Complement: Calculating And Converting For Binary Numbers Modern computers today use a binary number representation system called 'Two's complement'. It is a fixed number of binary digits used in computer calculations. Basic math operations such as addition and subtraction can be performed using the binary rules of addition and subtraction. Two's complement is not a complex scheme and is rather very simple to work with. This system also helps overcome the shortcomings of having to deal with magnitudes. Two's complement can be characterized as: 9. How to Calculate Cross Product and Rectangle Area in 3 Easy Steps Albeit the dot product of two vectors gives a scalar, in contrast, the resultant vector is a vector of cross-products. 10. Understanding Boolean Methods in Ruby With Examples Most programming languages have a Data Type called “Boolean”. This is
a form of data with only two possible values (usually “true” and
“false”). The Ruby language however does not have a Boolean Data
Type. 11. Divide and Conquer: Karatsuba Integer Multiplication Karatsuba algorithm's explanation with examples and illustrations. 12. Change Your Codes Behavior By Using Bitwise Operators Let’s dissect a weird bit flag program that took me a second to understand. in doing so, we’ll hopefully gain a more robust understanding of how bit masks and bitwise OR logic can manipulate values effectively. 13. Calculating the Square Root of a Number using the  Newton-Raphson Method [A How To Guide] Situations 14. What we need to know about Rounding and Midpoint Round a Number: Rounding a number means obtaining an approximate value that allows it to be used without cluttering up with useless precision for what we want to do, but in such a way that this approximation remains as close as possible to the exact value. 15. Python Tutorial: Recursive Function Debunked This Python Tutorial about the Recursive Function outlines the basic principles and methods to use recursion as a method of automating your software. 16. Motivation from Mathematics: Hire for Slope, not Y-Intercept Forget about the y-intercept; slope is the only thing that matters in the long run. 17. What is Collision Resistance in Cryptography? A short story about how national secrets, passwords, and financial data stays secret because of a small handful of smart people playing "Bet you can't" 18. AI And Machine Learning Are Our Best Bet To Keep Saving The World From Climate Catastrophe Many equations apply to Nuclear Fusion including the Maximum Entropy Principle. Fusion increases entropy. Think of unsolved equations relating to Nuclear Fusion as hardness problems. Whoever solves these problems or contributed towards software that solves these problems, helped achieve one of the biggest tasks in modern engineering and physics this century. I will be honest, many of us (including a certain somebody) want to win the race. Many of us are also removing obstacles from the obstacle course. 19. Display a Number's Prime Digits Program in java to input a number and display its prime digits. 20. Christmas Special: Games That Will Help You Understand Complex Math Concepts Games like chess and tic-tac-toe are games that have been with us for centuries and many people find them boring. What happens if they get really complicated? 21. Prefix Sums and How They Can be Used to Solve Coding Problems In this post, we will look at prefix sums and how they can be used to solve a common coding problem, that is, calculating the sum of an array (segment). This article will use Java for the code samples but the concept should apply to most programming languages. 22. Safeguarding Your Passwords from Cracking: Understanding the Math Concepts Passwords that seem safe to us are vulnerable to math-based hacking. This is why using concepts like Password Space may be able to make more secure passwords. 23. Java Program to Generate Multiplication Table of the Input Number Java program to display the multiplication of the input number from 1 to 10. 24. Understanding C++20 <chrono> in the Context of Quantitative Finance Modern C++20 chrono library usage in quantitative finance for fixed income securities pricing 25. A Clever Formula for Logarithm One of the best mathematical tools ever developed is the logarithm of a number. It has been used extensively in the past for the simplification of lengthy arithmetic calculations. The standard way of using the technique is via tables of common logarithms. The method of using these tables is well known and has been in use for decades. In this article, we will talk about a lesser known method of finding log of a number without using the tables. 26. Elliptic Curve Cryptography: A Basic Introduction Elliptic Curve Cryptography (ECC) is a modern public-key encryption technique famous for being smaller, faster, and more efficient than incumbents. 27. Natural Selection Vs. Intelligent Design: A Physicist's Perspective Our ability to come up with random numbers can reveal if we are part of a design in disguise. Are we? The age old question is in the reach of quantum biology. 28. Basic of  Cryptography What is Cryptography? 29. Must-Know Theorems for Programmers Programming is a complex and multifaceted field that encompasses a wide range of mathematical and computational concepts and techniques. 30. What Is Dynamic Programming? This article is for them, who have heard about Dynamic Programming and for them also, who have not heard but want to know about Dynamic Programming (or DP) . In this article, I will cover all those topics which can help you to work with DP . 31. Can’t Buy Me Love: An Argument for Implementing Illiquidity Mapping human values into code.  Hint: its hard to measure them in bitcoin. 32. Solving Matrix Algebra in JavaScript [An Overview] Many problems, especially in scientific computing, end up being formulated as matrix operations. This can be anything from PDE-solvers (PDE - Partial Differential Equation) up to image-processing or ML-algorithms. 33. How to Select a Random Node from a Tree Childhood moments with father 34. Distant Deep Teaching Abstract 35. Flax: Google’s Open Source Approach To Flexibility In Machine Learning Thinking of Machine Learning, the first frameworks that come to mind are Tensorflow and PyTorch, which are currently the state-of-the-art frameworks if you want to work with Deep Neural Networks. Technology is changing rapidly and more flexibility is needed, so Google researchers are developing a new high performance framework for the open source community: Flax. 36. Embed Github Gists in your Hacker Noon Stories The Hacker Noon editor now supports embedding Github gists. 37. Decoding Dynamic Programming In this post, we will solve a problem involving dynamic programming without being aware of it. 38. What is Zero-Knowledge Proof ? Let’s explain exactly what that term means in one sentence: 39. Fixed point math in Solidity It always seems impossible until it’s done. — Nelson Mandela 40. Introduction to Arithmetic Sequences An arithmetic sequence is a specific type of sequence in which the difference between two terms is “constant”. 41. Why Math Functions in C++ Are So Slow Why C++ math functions can be slow and how to fix them 42. Reasons Why Should Students Give More Emphasis on Derivatives and Integrals No wonder in saying that all students are not supposed to welcome the subject of Math with open arms. Additionally, when it comes to Calculus, the reasons for a nightmare for many would-be, Derivatives, and Integrals. Let narrow down the question of why students should give more emphasis on Derivatives and Integrals. 43. [Explained] Machine Learning Fundamentals: Optimization Problems and How to Solve Them If you start to look into machine learning and the math behind it, you will quickly notice that everything comes down to an optimization problem. Even the training of neural networks is basically just finding the optimal parameter configuration for a really high dimensional function. 44. SQL's 50 Year Reign: Here's Why SQL Is Still Relevant Today Dive into the detailed history of SQL's rise and reign over the past 50 years. They remained relevant by listening and adapting to the market. 45. Binary Lifting and Its Applications Binary Lifting and its use in finding Lowest Common Ancestor (LCA). Explore this amazing algorithm that speeds up ancestor queries in the tree data structure. 46. The REAL Life of Pi: Ethereal, Romantic, Mysterious And Completely Memorizing “Probably no symbol in mathematics has evoked as much mystery, romanticism, misconception and human interest as the number Pi” ~William L. Schaaf, Nature and History of Pi. π 47. Re: George Hotz, Simulation Theory, Stephen Wolfram, "God and Dice" When asked about the universe, its origin, and life in general, what comes to mind? The answer has fathomed scientist, theologians, philosophers and man for decades. I can go from the philosophical standpoint of being like, I am a Copt, here is what I believe. You and me, could go back and forth discussing what is truth, and all that. However, this isn't the point of this post. I want to look at the question of, "a universal theorem of everything". The whole universe and how it is tied in general is that of a physics standpoint. We are at the dawn of age of exploration really, and still mankind has a question on what unifies nature itself. 48. On Zero-Knowledge Proofs And Why They Matter There is a lot of enthusiasm for the application of zero-knowledge proof systems, at Dusk Network we’ve developed the open-source PLONK zero-knowledge proof system. It can be used for many things, and what better way to demonstrate its potential with an example! 49. Shakespeare Meets Google's Flax Some are born great, some achieve greatness, and some have greatness thrust upon them. William Shakespeare, Twelfth Night, or What You Will 50. Goldman Sachs, Data Lineage, and Harry Potter Spells Goldman Will Dominate Consumer Banking 51. Introduction To Maths Behind Neural Networks Today, with open source machine learning software libraries such as TensorFlow, Keras or PyTorch we can create neural network, even with a high structural complexity, with just a few lines of code. Having said that, the Math behind neural networks is still a mystery to some of us and having the Math knowledge behind neural networks and deep learning can help us understand what’s happening inside a neural network. It is also helpful in architecture selection, fine-tuning of Deep Learning models, hyperparameters tuning and optimization. 52. The Manual and Digital Ways of Calculating Factorization The factor is the numeral that can easily be divided into other numerals. It is to divide a numeral equally. If we have to divide 6, we are going to use 1, 2, 3, and 6. No other numerals can be used to divide it as they wouldn’t give the one numeral that is equally divisible. Factoring refers to the process of finding numbers so that results would be taken by multiplication. For this, we have to find out the highest common factor then divide it to have the result. 53. zk-SNARK Concepts Explained In Different Levels of Complexity I’ve decided to write an ELI15 for zk-SNARK jargon since I’ve yet to come across something similar during my studies. 54. Python Libraries For Data Science Top Data science libraries introduction of The Python programming language is assisting the developers in creating standalone PC games, mobiles, and other similar enterprise applications. Python has in excess of 1, 37,000 libraries which help in many ways. In this data-centric world, most consumers demand relevant information during their buying process. The companies also need data scientists for achieving deep insights by processing the big data. 55. Fenwick Tree Explained Fenwick Tree is an interesting data structure that uses binary number properties to solve point update and range queries in your code in some situations. 56. Introduction to Cryptography: Vigenère Cipher [Part 2] Hi again and welcome back to part 2 of the basic cryptography. Previously we discuss about what is cryptography and how fundamental and ancient problem it is in communication science. We talk about ceaser's cipher and how easy it is to break. We also talk about Substitution Cipher which was slight improvement to Ceaser' cipher but can easily be cracked using Frequency analysis technique. 57. Tech Politics, Math, and Career Baakt Launch 58. The Digital Universal Drachma: A Paradigm for a Global Private-Public Cryptocurrency A bold re-envisioning of the fundamentals behind blockchain technology and economic theory, in this paper Chitose Nakamoto suggests a middle ground in crypto. 59. Math GPT: Can AI Help Solve Unified Theory ? What if we trained AI to complete equations instead of images of Cats? 60. The Concept of Numbers for a Y-Generation Organic Processing Unit Whether it's consciously or subconsciously, we use numbers every single moment of our lives. Numbers help us navigate in what we refer to as real life, as they help us set fixed determinations of concepts that are impossible to comprehend otherwise, eliminating the need for "lagging." 61. 5 Ways to Get Kids Interested in Stem Studying STEM has become crucial for students who want to be part of an innovative, rapidly growing industry with excellent career prospects. 62. How I Built a Simple Report Card Program in Java for 15 Subjects Part 2 Write a program to take marks of 15 subjects from the user and display the Total, Percentage, Highest Marks, Average, and Remarks. 63. Introduction To The Convolution In this article, we are going to learn about the grayscale image, colour image and the process of convolution. 64. How to Kalman Filter Your Way Out Learn how to use Kalman filters to minimize uncertainty with multi-sensory arrays 65. Learning AI If You Suck at Math — P6 — Math Notation Made Easy! If you’ve followed parts 1, 2, 3, 4, 5 and 7 of this series you know that you really don’t need a lot of math to get started with AI. You can dive right in with practical tutorials and books on the subject. 66. Pseudo-Mathematical Proof of All Things Being Skills You Can Get Better At 67. The Mind-Bending World of Heuristic Emergence Might Mean We're Artificial Imagine winning the lottery! 68. VR Math: The Difficulties of Moving Objects in Virtual Reality Applied game dev math can be tricky, but is really rewarding. Here’s a follow-up to Nevyn’s Twitter thread on his journey from incomprehensible mess to enlightened one-liner. 69. The Mathematical Analysis of Logic by George Boole - Table of Links The Mathematical Analysis of Logic, by George Boole, is part of the HackerNoon Books Series. Read this book online for free on HackerNoon! 70. Introduction to Threshold Signatures Today we’re announcing cross-chain zaps integration into the Symbiosis protocol. 71. A Brief Introduction to Algorithmic Complexity It’s not just the running time; it’s the space usage too. We see algorithms used in pretty much every program that’s larger than a college project. 72. I Really Struggled With Math Because My Teachers Were Horrible I really struggled with Math at a point cause my teachers were horrible. 73. Geometrical Constants in Equilateral Triangles: Part I Introduction 74. The Gambler's Ruin Explained 10-K Diver explains the gambler's ruin. 75. What We Need to Know About Standard Deviation & Distance Formula Standard Deviation Elucidation: Standard deviation is particularly used to catch the values of dispersed data of a set. It shows the result in two states, whether the answer lies near or away from the reference or mean position.  It is important to realize that any Regular Distribution includes a Mean and Standard Deviation. The Regular Distribution is symmetrical about the Mean. The Standard Deviation determines the distribution's shape. 76. A 42-Min Fall: A Physicist's Take on Falling Through The Centre of The Earth The classic 1864 science fiction novel by Jules Verne, titled "Journey to the Center of the Earth"  has mesmerized many generations. The desire to discover the secrets inside the earth is perhaps as old as the desire to discover the secrets  of the stars. 77. Thrilled to be Recognized as The HackerNoon Contributor of the Year - MATHEMATICS Merlyn is the teacher’s digital assistant, a multimodal solution built to give time back to teachers. 78. An Insecure Pillar in Cybersecurity From Caesar's cipher to RSA. A deep dive into the inner workings of modern cryptography and its potential crumbling. 79. The Truth About Less Biased Data-Informed Predictive Policing Critics say it merely techwashes injustice 80. The Story of Machine Proofs - Part II Automated theorem proving, 81. How to Kalman Filter Your Way Out (Part 2: Updating Your Prediction) Part II describes how to use Kalman filters to minimize uncertainty when using multi-sensor arrays 82. Breaking Down Secretary Problem What did you think of when you had a crush on someone? Did you fantasize about marriage with him/her? When you were in some serious relationship, did you plan marriage with your partner? How did the relationship turn out? Some relationships turn into a marriage, and some don’t. Hearing stories of many friends, I see extremely few people being in a relationship (and later marrying) with only one person whole over their life. 83. Deep Learning for Modeling Audio-Visual Correspondences Human perception is multidimensional and a balanced combination of hearing, vision, smell, touch, and taste. Recently, many pieces of research have tried to step forward on the road of improving machine perception by transitioning from single-modality learning to multimodality learning. 84. Art + Science + Scripture = Bible Math This science/scripture/art article known as Bible Math came about in a most-riveting, creative, and spiritual way. For over two decades, I have been leading weekly Bible Discussion Groups and studying the Bible with people on an individual mentoring level. In the course of these spiritual adventures, as part of the City of Angels International Christian Church, I am typically delighted to share the gospel with all who are willing to listen and put the word of God into practice. Such delight is especially true of the following mathematics-focused Bible study. 85. Object Detection Using Single Shot MultiBox Detector (A Case Study Approach) This blog post delivers the fundamental principles behind object detection and it's algorithms with rigorous intuition. Thank you for checking out the 85 most read stories about Mathematics on HackerNoon. to find the most read stories about any technology. Visit the /Learn Repo

85 Stories To Learn About Mathematics

THOUGHTS ON SCIENCE

Image credit: on Shubham Dhage Unsplash It's no secret that large language models (LLMs) like ChatGPT have transformed how we work today. The trading landscape is no different. Crypto With the introduction of these powerful and new technologies, Crypto traders have thought of various ways they can use these technologies to revolutionize the way traders approach market analysis, generate trading signals, automate trading processes, and even democratize decision-making. In this article, I will mention some ways LLMs can be used to change the way we trade and potentially reshape the trading landscape: Sentiment analysis based trading Strategy LLM models, which are deep learning algorithms, can perform various tasks. One of these tasks is natural language processing (NLP) tasks. This means they can process large volumes of textual data from news articles, social media, and other sources to provide a comprehensive market analysis that can be used for sentiment analysis-based trading. This can save time for traders as they no longer need to sift through large volumes of news sources. So, how does this work? A sentiment analysis-based trading strategy allows traders to gauge the market sentiment of some cryptocurrencies and identify market-moving events. Traders can use the result from this sentiment for market trends analysis, allowing traders to swiftly react to news and decide when to buy and sell. Zanyang talked a bit more about this in his , where he used algorithmic trading platforms like QuantConnect. Use of NLP-Powered sentiment analysis in trading strategy paper Customer support and communication Besides Traders, LLMs can benefit everyone in the crypto space, including cryptocurrency founders, project owners, community leaders, and trading platforms. For community and trading platforms, LLMs can be used for knowledge-based databases, Q&A bots, and customer support. This way, they can seamlessly help users navigate the platform, improve customer experience, answer project-related questions, resolve issues, and assist in technical support. The best thing about this system is that, unlike traditional conversational AI, LLM-powered bots can understand human language and respond better 24/7. Some AI Chatbots you can explore for this are and . AwesomeQA UltimateGPT Generate trading signals A trading signal is a trigger telling a trader when to buy or sell orders. Triggers are generated by simply analyzing indicators, patterns, or data points. These common technical indicators include MACD, RSI, KD, and Bollinger Bands. This talks a lot more about these indicators. article by BYDFi With LLMs, you can generate trading signals in various ways: Building trading algorithms that depend on pre-established rules. Identify trends through sentiment. Interprets news events that can cause market movements. Recognize patterns and interpret news events. With these signals, traders can use the insights to capitalize on anticipated price reactions and make more informed decision-making. The best part is that this signal generation can be automated, thus reducing any latency in trade execution. This exciting discussion on the . Numerai forum discusses using LLMs to create trading signals Build, Build, and Build with LLMs You’ve probably used LLMs in code generation or to assist you in generating codes; If not, I have got an [Analytics Vidhya’s article on How to Build LLMs for Code?](https://www.analyticsvidhya.com/blog/2023/09/llms-for-code/#:~:text=A%20Large%20Language%20Model%20(LLM,understand%20and%20generate%20computer%20code.) for you. Based on your prompt, LLM can be used to write code faster and much more efficiently. You can also use it to generate a whole project. Bringing this idea to the crypto space, you can use LLM models like , , and to generate codes, write smart contracts, and test. These models can also read, analyze and modify your codes or even improve your codebase. WizardCoder StarCoder Codeium All this can be done using simple prompts and knowing what you want. Democratizing decision-making and knowledge One of my favorite things about LLM models is that they make information accessible. You can get information about just about anything by writing a simple prompt. Imagine a model trained on real-world financial data that can understand financial concepts, formulas, and market data. This means you can ask this model about anything in this domain, and it will provide you with information and help you make better decisions. Such a custom LLM would benefit not just market leaders but novice traders and beginners in the crypto space. One of my favorite Crypto knowledge bases is , an AI-powered search engine. MinMax Wrap-up You can use LLMs in various ways, from generating codes to providing 24/7 customer support. While many of these models are still under development, they are becoming increasingly powerful and more accurate as the day passes. As this happens, you will likely use them for even more tasks in the crypto industry in the future.

How LLMs like ChatGPT Can Change the Way We Trade

Things To Make by Archibald Williams is part of the HackerNoon Books Series. You can jump to any chapter in this book . A HOME-MADE PANTOGRAPH here XXXVI. A HOME-MADE PANTOGRAPH. The pantograph is a simple apparatus for copying drawings, maps, designs, etc., on a reduced or enlarged scale, or to the same size as the original. [Illustration: FIG. 186.—Details of simple pantograph.] A sketch of a pantograph is given in Fig. 186. Four rods are jointed together to form a parallelogram, the sides of which can be lengthened or shortened to suit the scale of reproduction. One is attached by a fixed pivot at a to the board on which the drawing is done. At b and e are removable pivots, used for adjusting the rods; at c is a pivot which projects an inch or so below the rods. The pointer is inserted at d for enlargement, or at f for reduction, the pencil being in the unoccupied hole at d or f. If a same-sized copy is desired, the fixed pivot is transferred to d, and the pencil and pointer placed at a and f respectively. Construction of an Enlarging and Reducing Pantograph.—Cut out of 1/8-inch oak, walnut, or beech four rods 5/8 inch wide and 19 inches long. Smooth them well all over, and make marks near the ends of each, exactly 18 inches apart. The graduation of the rods for the adjustment pivot holes is carried out in accordance with the measurements given in Fig. 187. It is advisable to mark out and bore each rod separately if you do not possess a machine which will drill holes quite perpendicularly; if you do, all four rods can be drilled at one operation. In Fig. 187 the lower row of numerals indicates the number of times (in diameters) the original is enlarged when all four holes similarly figured are used; the upper row, the size of the copy as compared with the original in case of reduction. If proportions other than those given are required, a very little calculation will locate the necessary holes. Pivots.—All the pivots must fit their holes accurately, as any looseness at the joints detracts from the truth of reproduction. For pivots band b and e may use brass screws and small pieces of hard wood as nuts to hold them in position. The nuts should screw on rather stiffly, and not be forced hard against the rods, as free motion with little friction at all joints is essential for good work. [Illustration: FIG. 187.—Diagram showing how to mark off pantograph rods. The dotted lines above rod give distances of holes from ends.] The fixed pivot at a may be merely the shank of a wire nail of the proper size driven into the board, a cork collar being slipped over it to keep the rod the proper distance from the board. For c use a screw to the head of which has been soldered half an inch of a round-headed brass nail, which will move easily over the paper. At d is needed a hollow pivot, fashioned out of a quarter of an inch of pencil-point protector or some other thin tube, burred over slightly at the ends so as not to fall out. The end of B at f has a slotted hole to grip the pencil or pointer, as the case may be. A Same-size Pantograph.—For making a same-size copy, tracing may be preferred to the use of a pantograph; but if a pantograph is adopted, a special apparatus may be constructed for the purpose. The arrangement is exactly the same as that already described, excepting that the only holes needed are those at a, c, d, f, at the middle points of the four rods, the parallelogram formed by the rods being equal-sided. The fixed pivot is situated at d, and pencil and pointer holes are made at a and f. Using the Pantograph.—When adjusting the instrument for reduction or enlargement, make sure that the adjustment pivots are in the holes corresponding with the scale. The fixed pivot, pointer, and pencil must be rigid, and, with pivot c, be of such a length that the pantograph as a whole moves parallel to the paper. A little sliding weight to place on the rod near the pencil will be found useful for keeping the pencil point in constant contact with the paper. If the apparatus works stiffly, ease the holes a trifle and lead-pencil the wood at all points where two surfaces rub. It is absolutely impossible to make a good reproduction with a stiff, jerky pantograph. To decide the positions of original and the paper for the copy, get the pointer centred on the original and adjust the paper till its centre is under the pencil. About HackerNoon Book Series: We bring you the most important technical, scientific, and insightful public domain books. This book is part of the public domain. Archibald Williams (2005). Things To Make. Urbana, Illinois: Project Gutenberg. Retrieved https://www.gutenberg.org/cache/epub/14664/pg14664-images.html This eBook is for the use of anyone anywhere at no cost and with almost no restrictions whatsoever. You may copy it, give it away or re-use it under the terms of the Project Gutenberg License included with this eBook or online at , located at . www.gutenberg.org https://www.gutenberg.org/policy/license.html

A HOME-MADE PANTOGRAPH

The Romance of Modern Mechanism by Archibald Williams is part of the HackerNoon Books Series. You can jump to any chapter in this book . CHAPTER XXVI here SCULPTURING MACHINES The savage who, with a flint point or bone splinter, laboriously scratched rude figures on the walls of his cave dwelling, did the best he was capable of to express the emotions which affect the splendidly equipped sculptor of to-day; he wished to record permanently some shape in which for the time he was interested, religiously or otherwise. The sun, moon and stars figure largely in primitive religions as objects of worship. They could be easily suggested by a few strokes of a tool. But when mortals turned from celestial to terrestrial bodies, and to the worship of human or animal forms—the "graven images" of the Bible—a much higher level of art was reached by the sculptor, who endeavoured to give faithful representations in marble of the great men of the time and of the gods which his nation acknowledged. The Egyptians, whose colossal monuments strew the banks of the Nile, worked in the most stubborn materials—basalt, porphyry and granite—which would turn the edge of highly tempered steel, and therefore raise wonder in our minds as to the nature of the tools which the subjects of the Pharaohs must have possessed. Only one chisel, of a bronze so soft that its edge turned at the first stroke [336]against the rock under which it was found, has so far come to light. Of steel tools there is no trace, and we are left to the surmise that the ancients possessed some forgotten method of hardening other metals—including bronze—to a pitch quite unattainable to-day. Whatever were their implements, they did magnificent work; witness the splendid sculptures of vast proportions to be found in the British Museum; and the yet huger statues, such as those of Memnon and those at Karnak, which attract tourists yearly to Egypt. The Egyptians admired magnitude; the Greeks perfection of outline. The human form in its most ideal development, so often found among a nation with whom athleticism was almost a religion, inspired many of the great classical sculptors, whose work never has been, and probably never will be, surpassed. Great honour awaited the winner in the Olympian games; but the most coveted prize of all was the permission given him—this after a succession of victories only—to erect a statue of himself in the sacred grove near the shrine of Olympian Jove. Happy the man who knew that succeeding generations would gaze upon a marble representation of some characteristic attitude assumed by him during his struggle for the laurel crown. Until recently the methods of sculpture have remained practically unaltered for thousands of years. The artist first models his idea in clay or wax, on a small scale. He then, if he designs a life-size or colossal statue, erects a kind of iron skeleton to carry the clay of the full-sized model, copied proportionately from the smaller one. When this is finished, a piece-mould is formed from it by applying wet lumps of plaster of Paris all over the surface in such a manner that they can be removed piecemeal, and fitted together to form a complete mould. Into this liquid plaster is run, for a hollow cast of the whole figure, which is smoothed and given its finishing touches by the master hand. This cast has next to be reproduced in marble. Both the cast and the block of marble are set up on "scale-stones," revolving on vertical pivots. An ingenious instrument, called a "pointing machine," now comes into play. It has two arms ending in fine metal points, movable in ball-and-socket joints. These arms are first applied to the model, the lower being adjusted to touch a mark on the scale-stone, the upper to just reach a mark on the figure. The operator then clamps the arms and revolves the machine towards the block of marble, the scale-stone of which has been marked similarly to its fellow. The bottom arm is now set to rest on the corresponding mark of the scale-stone; but the upper, which can slide back telescopically, is prevented from assuming its relative position by the unremoved portions of the block. The workman therefore merely notices the point on the block at which the needle is directed, and drills a hole into the marble on the line of the needle's axis, to a depth sufficient to allow the arm to be fully extended. This process is repeated, in some cases many thousands of times, until the block has been honeycombed with small holes. The carver can now strike off the superfluous marble, never going beyond the depth of a hole; and a rough outline of the statue appears. A more skilled workman follows him to shape the material to a close copy of the cast; and the [338]sculptor himself adds the finishing touches which stamp his personality on the completed work. Only a select few of the world's greatest sculptors have ventured to strike their statues direct from the marble, without recourse to a preliminary model. Such a one was Michelangelo, who, as though seized by a creative frenzy, would hew and hack a block so furiously that the chips flew off like a shower, continuing his attack for hours, yet never making the single false stroke that in the case of other masters has ruined the work of months. He truly was a genius, and must have possessed an almost supernatural faculty of knowing when he had reached the exact depth at any point in the great block of marble from which his design gradually emerged. The formation of artistic will always require the master's hand; but the of the cast in marble or stone can now be performed much more expeditiously than is possible with the pointing machine. We have already two successful mechanisms which in an almost incredibly short time will eat a statue out of a block in faithful obedience to the movement of a pointer over the surface of a finished design. They are the Wenzel Machine Sculptor and Signor Augusto Bontempi's . models reproduction Meccaneglofo THE WENZEL SCULPTURING MACHINE In the basement of a large London business house we found, one dark November afternoon, two men at work with curious-looking frameworks, which they swayed backwards and forwards, up and down, to the accompaniment of a continuous clattering of metal upon stone. Approaching nearer, we saw, lying horizontally in the centre of the machine, a small marble statue, its feet clamped to a plate with deep notches in the circumference. On either side, at equal distances, were two horizontal blocks of marble similarly attached to similar plates. The workman had his eyes glued on a blunt-nosed pointer projecting from the middle of a balanced frame. This he passed slowly over the surface of the statue, and simultaneously two whirring drills also attached to the frame ate into the stone blocks just so far as the movement of the frame would permit. The drills were driven by electric power and made some thousands of revolutions per minute, throwing off the stone they bit away in the form of an exceedingly fine white dust. It was most fascinating to watch the almost sentient performance of the drills. Just as a pencil in an artist's hands weaves line into line until they all suddenly spring into life and show their meaning, so did the drills chase apparently arbitrary grooves which united, spread, and finally revealed the rough-hewn limb. Every now and then the machinist twisted the footplates round one notch, and snicked the retaining bolts into them. This exposed a fresh area of the statue and of the blocks to the pointer and the drills. The large, coarse drills used to clear away the superfluous material during the earlier stages of the work were replaced by finer points. The low relief was scooped out, the limbs moulded, the delicate curves of cheek and the pencilling of eyebrows and lips traced, and in a few hours the copies were ready for the usual smoothing and finishing at the hands of the human sculptor. According to the capacity of the machine two, four, or six duplicates can be made at the cost of a little more power and time. Nor is it necessary to confine operations to stone and marble, for we were shown some admirable examples of wooden statues copied from a delicate little bronze, and, were special drills provided, the relations could be reversed, bronze becoming passive to motions controlled by a wooden original. "Sculpturing made easy" would be a tempting legend to write over the Wenzel machine. But it would not represent the truth. After all, the mechanism only , it cannot originate, which is the function of the sculptor. It stands to sculpturing in the same relation as the printer's "process block" to the artist's original sketch, or the lithographic plates to the painter's coloured picture. Therefore prejudice against machine-made statues is as unreasonable as objection to the carefully-executed of a celebrated painting. The sculptor himself has not produced it at first hand, yet his personality has been stamped even on the copy, for the machine can do nothing except what has already been done for it. The machine merely displaces the old and imperfect "pointing" by hand, substituting a method which is cheaper, quicker, and more accurate in its interpretation of the model. copies replica It is obvious that, apart from sculpture proper, the industrial arts afford a wide field for this invention. In architecture, for instance, carved wood and stonework for interiors and exteriors of buildings have been regarded hitherto as expensive luxuries, yet in spite of their cost they are increasingly indulged in. The architect now has at his disposal an economical method of carving which will enable him to utilise ornamental stonework to almost any degree. Sculptured friezes, cornices, and capitals, which, under the old régime, would represent months of highly paid hand labour, may now be reproduced rapidly and in any quantity by the machine, which could be adapted to work on the scaffolding itself. What will become of the stonemasons? Won't they all be thrown out of work, or at least a large number of them? The best answer to these questions will be found in a consideration of industries in which machinery has replaced hand work. Has England, as a cotton-spinning nation, benefited because the power-loom was introduced? Does she employ more operatives than she would otherwise have done, and are these better paid than the old hand weavers? All these queries must have "Yes!" written against them. In like manner, if statuary and decoration becomes inexpensive, twenty people will be able to afford what hitherto was within the reach of but one; and an industry will arise beside which the output of the present-day monumental mason will appear very insignificant. The sculpturing machine undoubtedly brings us one step nearer the universal House Beautiful. A complete list of the things which the versatile "Wenzel" can perform would be tediously long. Let it therefore suffice to mention boot-lasts, gun-stocks, moulds, engineering patterns, numeral letters, and other articles of irregular shape, as some of the more prosaic productions which grow under the buzzing metal points. Some readers may be glad to hear that the Wenzel promises another hobby for the individual who likes to "use his hands," since miniature machines are purchasable which treat subjects of a size not exceeding six inches in diameter. No previous knowledge of carving is necessary, and as soon as the elementary principles have been mastered the possessor of a small copier can take advantage of wet days to turn out statuettes, busts, and ornamental patterns for his own or friends' mantelpieces. And surely a carefully finished copy in white marble of some dainty classic figure or group will be a gift well worth receiving! The amateur photographer, the fret-sawyer, and the chip-carver will have to write "Ichabod" over their workshops! The Wenzel has left its experimental stage far behind. The German Emperor, after watching the creation of a miniature bust of Beethoven, expressed his delight in a machine that could call a musician from lifeless stone. The whole of the interior decoration of the magnificent Rathaus, Charlottenburg, offers a splendid example of mechanical wood carving, which tourists would do well to inspect. We may now pass to THE BONTEMPI SCULPTURING MACHINE, for such is the translation of the formidable word . This machine is the invention of Signor Augusto Bontempi, a native of Parma, who commenced life as a soldier in the Italian army, and while still young has won distinction as a clever engineer. Meccaneglofo His machine differs in most constructional details from the Wenzel. To begin with, the pressure of the drills on the marble is imparted by water instead of by the hand; secondly, the block to be cut is arranged vertically instead of horizontally; thirdly, the index-pointer is not rigidly connected to the drill frame, but merely controls the valves of hydraulic mechanism which guides the drills in any required direction. The drills are by electricity, but all their other movements come from the pressure of water. rotated Undoubtedly the most ingenious feature of the Bontempi apparatus is the pointer's hydraulic valve, which gives the drills a forward, lateral, or upward movement, or a compound of two or three movements. When the pointer is not touched all the valve orifices remain closed, and the machine ceases to work. Should the operator pull the pointer forwards a water-way is opened, and the liquid passes under great pressure to a cylinder which pushes the drill frame forward. If the pointer be also pressed sideways, a second channel opens and brings a second cylinder into action, and the frame as a whole is moved correspondingly, while an upward twist operates yet a third set of cylinders, and the workman himself rises with the drills. As soon as the sensitive tip of the pointer touches an object it telescopes, and immediately closes the valves, so that the drills bore no further in that direction. The original and copies are turned about from time to time on their bases in a manner similar to that already described in treating the Wenzel. As many as twenty copies can be made on the largest machines. Quite recently there has been installed in Southwark, London, a gigantic Bontempi which stands 27 feet high, and handles blocks 5 feet 6 inches square by 10 feet high, and some 20 tons in weight. Owing to the huge masses to be worked only one copy can be made at a time; though, doubtless, if circumstances warranted the expense, a machine could be built to do double, triple, or quadruple duty. The proprietors have discovered an abrasive to grind granite—ordinary steel chisels would be useless—and they expect a great demand for columns and monumental work in this stubborn material, as their machines turn out finished stuff a dozen times faster than the mason. An interesting story is told about the early days of Signor Bontempi's invention. When he set up his experimental machine at Florence, the workmen, following the example of the Luddites, rose in a body and threatened both him and his apparatus with destruction. The police had to be called in to protect the inventor, who thought it prudent to move his workshop to Naples, where the populace had broader-minded views. The Florentines are now sorry that they drove Signor Bontempi away, for they find that instead of depressing the labour market, the mechanical sculptor is a very good friend to both proprietor and employé. Note.—For information and illustrations the author has to thank Mr. W. Hanson Boorne, of the Machine Sculpture Company, Aldermary House, London, E.C., and Mr. E. W. Gaz, secretary of the Automatic Sculpture Syndicate, Sumner Street, Southwark. About HackerNoon Book Series: We bring you the most important technical, scientific, and insightful public domain books. This book is part of the public domain. Archibald Williams (2014). The Romance of Modern Mechanism. Urbana, Illinois: Project Gutenberg. Retrieved https://www.gutenberg.org/cache/epub/46094/pg46094-images.html. This eBook is for the use of anyone anywhere at no cost and with almost no restrictions whatsoever. You may copy it, give it away or re-use it under the terms of the Project Gutenberg License included with this eBook or online at , located at . www.gutenberg.org https://www.gutenberg.org/policy/license.html

Stories, News & Mentions about NUMERADE

Numerade’s Stories on HackerNoon