A new approach to healthcare. Breaking barriers to bring healthcare directly to you.

We are an Australian owned and women-led healthcare company that specializes in weight loss through telehealth and video consultations. Our team of experienced healthcare professionals, including doctors and nurse practitioners, work with patients to develop individualized treatment plans that are tailored to their specific needs and goals. Our focus is on providing high-quality, personalized care that empowers patients to take control of their health and achieve long-lasting results. We are committed to leveraging the latest technology to make healthcare more accessible and convenient for everyone. At our core, we believe in delivering compassionate care that is grounded in science and driven by innovation.

Nume

This paper is available on arxiv under CC 4.0 license.
Authors:
(1) Jihoon Chung;
(2)  Zhenyu (James) Kong. Table of Links Abstract & Introduction
Review of Related Work
Proposed Research Methodology
Numerical Case Studies
Real-World Simulation Case Studies
Conclusion
References IV. NUMERICAL CASE STUDIES This section provides numerical studies to compare the performance between the proposed method and benchmark methods. The studies evaluate the performance by varying the correlations (ki in Eq. (7)) between the KCCs in nonstationary process faults. All the numerical case studies consist of 20 independent trials. The code of the proposed algorithm is implemented in Matlab 2021. The CPU used in case studies is an Intel® Core™ Processor i7-8750H. This section compares the proposed method with the following benchmark methods. • User Grouping Sparse Bayesian Learning (i.e., UGSBL) proposed in [2] is an SBL method that clusters the KPCs samples into groups sharing the same process faults in sparse estimation. • Spatially Correlated Bayesian Learning (i.e., SCBL) proposed in [8] is an SBL method to diagnose process faults by exploiting the spatial correlation of KCCs. • [10] proposed the SBL method that considers prior knowledge of the process faults in sparse estimation. • MSBL proposed in [32] is a basic SBL method that assumes the independence of KCCs and stationarity of process faults. Performance Evaluation: To evaluate the effectiveness of the proposed method, two performance measures are used in this paper. The first measure evaluates the process fault detection capability, and the second measure is related to estimation accuracy. Each measure is evaluated by varying the correlation coefficients of spatially correlated KCCs. Fault Detection Capability: First, the proposed method requires correctly identifying process faults whose KCCs have excessive variance. In order to assess the capability of detecting process faults under sparse conditions, the area under the receiver operating characteristics (ROCs) curve (AUC) is utilized [46]. AUC is widely used for evaluating the accuracy of binary classification. Specifically, AUC measures the quality of the classification accuracy between process faults and non process faults irrespective of the threshold of variance. In other words, AUC accounts for the trade-off between the type 1 and type 2 errors. When the method is capable of achieving perfect classification results, the resulting AUC value will be 1. However, the AUC with a value close to 0.5 denotes that the estimated variances of KCCs from the method are similar between the process faults and non-process faults. Table I represents that the proposed method always achieves the best performance of AUC in the various spatial correlation between KCCs. Specifically, the AUC of the proposed method is close to 1.0 at every correlations levels, representing the perfect classification between the process faults and nonprocess faults in terms of estimated variance. Since UGSBL clusters the KPCs samples that share the same process faults, the method shows a comparable result to the proposed method  when the spatial correlation of KCCs is relatively low and moderate. However, the performance degrades in the high spatial correlation since the method assumes independence between all KCCs. The performance of SSBL is robust to correlations. This is because SSBL utilizes the correlation of correlated KCCs in its estimation. However, SSBL performs worse than the proposed method since this method assumes the same process faults between all KPCs samples, which are unsuitable for dealing with nonstationary process faults. MSBL and [10] show poor performances since these methods assume the independence of KCCs and stationarity of process faults. 2) Estimation Accuracy: In addition to correctly identifying the process fault, the method needs to accurately estimate the variance of process faults. After the process faults are identified, the practitioners are interested in how large the variances are to make decisions for the maintenance of the process. To achieve this objective, this paper provides the normalized mean squared error (NMSE) between true and estimated variance. Table II represents that the proposed method achieves the best performance in the estimation accuracy of the variance in most of the correlations. It implies that the proposed method is useful not only for the process faults identification but also for measuring how severe the process faults are. Compared to benchmark methods, the effectiveness of the proposed method in the NMSE results is noticeable in the presence of high spatial correlations between KCCs. Unlike AUC performance, the NMSE of SSBL increases as the correlation increases. It implies that SSBL successfully identifies the process faults utilizing the correlation information but cannot accurately estimate the variance since SSBL assumes the stationary process faults over KPC samples. The performance of UGSBL, MSBL, and [10] represent similar trends to the performance of AUC. This paper is available on arxiv under CC 4.0 license. Authors: (1) Jihoon Chung; (2)  Zhenyu (James) Kong. This paper is available on arxiv under CC 4.0 license. Authors: (1) Jihoon Chung; (2)  Zhenyu (James) Kong. Table of Links Abstract & Introduction Review of Related Work Proposed Research Methodology Numerical Case Studies Real-World Simulation Case Studies Conclusion References Abstract & Introduction Abstract & Introduction Review of Related Work Review of Related Work Proposed Research Methodology Proposed Research Methodology Numerical Case Studies Numerical Case Studies Real-World Simulation Case Studies Real-World Simulation Case Studies Conclusion Conclusion References References IV. NUMERICAL CASE STUDIES This section provides numerical studies to compare the performance between the proposed method and benchmark methods. The studies evaluate the performance by varying the correlations (ki in Eq. (7)) between the KCCs in nonstationary process faults. All the numerical case studies consist of 20 independent trials. The code of the proposed algorithm is implemented in Matlab 2021. The CPU used in case studies is an Intel® Core™ Processor i7-8750H. This section compares the proposed method with the following benchmark methods. • User Grouping Sparse Bayesian Learning (i.e., UGSBL) proposed in [2] is an SBL method that clusters the KPCs samples into groups sharing the same process faults in sparse estimation. • Spatially Correlated Bayesian Learning (i.e., SCBL) proposed in [8] is an SBL method to diagnose process faults by exploiting the spatial correlation of KCCs. • [10] proposed the SBL method that considers prior knowledge of the process faults in sparse estimation. • MSBL proposed in [32] is a basic SBL method that assumes the independence of KCCs and stationarity of process faults. Performance Evaluation: To evaluate the effectiveness of the proposed method, two performance measures are used in this paper. The first measure evaluates the process fault detection capability, and the second measure is related to estimation accuracy. Each measure is evaluated by varying the correlation coefficients of spatially correlated KCCs. Performance Evaluation: Fault Detection Capability: First, the proposed method requires correctly identifying process faults whose KCCs have excessive variance. In order to assess the capability of detecting process faults under sparse conditions, the area under the receiver operating characteristics (ROCs) curve (AUC) is utilized [46]. AUC is widely used for evaluating the accuracy of binary classification. Specifically, AUC measures the quality of the classification accuracy between process faults and non process faults irrespective of the threshold of variance. In other words, AUC accounts for the trade-off between the type 1 and type 2 errors. When the method is capable of achieving perfect classification results, the resulting AUC value will be 1. However, the AUC with a value close to 0.5 denotes that the estimated variances of KCCs from the method are similar between the process faults and non-process faults. Fault Detection Capability: First, the proposed method requires correctly identifying process faults whose KCCs have excessive variance. In order to assess the capability of detecting process faults under sparse conditions, the area under the receiver operating characteristics (ROCs) curve (AUC) is utilized [46]. AUC is widely used for evaluating the accuracy of binary classification. Specifically, AUC measures the quality of the classification accuracy between process faults and non process faults irrespective of the threshold of variance. In other words, AUC accounts for the trade-off between the type 1 and type 2 errors. When the method is capable of achieving perfect classification results, the resulting AUC value will be 1. However, the AUC with a value close to 0.5 denotes that the estimated variances of KCCs from the method are similar between the process faults and non-process faults. Table I represents that the proposed method always achieves the best performance of AUC in the various spatial correlation between KCCs. Specifically, the AUC of the proposed method is close to 1.0 at every correlations levels, representing the perfect classification between the process faults and nonprocess faults in terms of estimated variance. Since UGSBL clusters the KPCs samples that share the same process faults, the method shows a comparable result to the proposed method  when the spatial correlation of KCCs is relatively low and moderate. However, the performance degrades in the high spatial correlation since the method assumes independence between all KCCs. The performance of SSBL is robust to correlations. This is because SSBL utilizes the correlation of correlated KCCs in its estimation. However, SSBL performs worse than the proposed method since this method assumes the same process faults between all KPCs samples, which are unsuitable for dealing with nonstationary process faults. MSBL and [10] show poor performances since these methods assume the independence of KCCs and stationarity of process faults. 2) Estimation Accuracy: In addition to correctly identifying the process fault, the method needs to accurately estimate the variance of process faults. After the process faults are identified, the practitioners are interested in how large the variances are to make decisions for the maintenance of the process. To achieve this objective, this paper provides the normalized mean squared error (NMSE) between true and estimated variance. Table II represents that the proposed method achieves the best performance in the estimation accuracy of the variance in most of the correlations. It implies that the proposed method is useful not only for the process faults identification but also for measuring how severe the process faults are. Compared to benchmark methods, the effectiveness of the proposed method in the NMSE results is noticeable in the presence of high spatial correlations between KCCs. Unlike AUC performance, the NMSE of SSBL increases as the correlation increases. It implies that SSBL successfully identifies the process faults utilizing the correlation information but cannot accurately estimate the variance since SSBL assumes the stationary process faults over KPC samples. The performance of UGSBL, MSBL, and [10] represent similar trends to the performance of AUC.

Sparse Bayesian Learning: Numerical Case Studies

Efficient Neural Network Approaches for Conditional Optimal Transport: Numerical Experiments

This paper is under CC 4.0 license. available on arxiv Authors: (1) Amir Noorizadegan, Department of Civil Engineering, National Taiwan University; (2) D.L. Young, Core Tech System Co. Ltd, Moldex3D, Department of Civil Engineering, National Taiwan University & dlyoung@ntu.edu.tw; (3) Y.C. Hon, Department of Mathematics, City University of Hong Kong; (4) C.S. Chen, Department of Civil Engineering, National Taiwan University & dchen@ntu.edu.tw. Table of Links Abstract & Introduction Neural Networks PINN for Solving Inverse Burgers’ Equation Residual Network Numerical Results Results, Acknowledgments & References 5 Numerical Results 1. Mean Square Error: The training errors shown in the plotted graphs, relative to the iteration number, are computed using the mean square error criterion. 2. Relative L2 Norm Error: The validation errors, calculated over the test data and presented in the plotted graphs concerning the iteration number, are measured using the relative L2 norm error metric. 3. Maximum Absolute Error: When visualizing errors across the entire domain, whether in 2D or 3D scenarios, the error represented on the contour error plot is referred to as the maximum absolute error. It is important to note that the contour bars are scaled according to the largest error in the plot. Maximum Absolute Error = max |u − N| These error metrics provide valuable insights into the accuracy and convergence of the methods used in this study. In this section four methods will be investigated. 1. Plain NN: A conventional neural network without any additional modifications or residual connections (see Fig. 2(a)). 2. ResNet: A residual neural network architecture where the output of each layer is obtained by adding the residual to the layer’s output (see Fig. 2(b)). 3. SkipResNet: An extension of ResNet, where the residual connection is applied every other layer, alternating between including and excluding the residual connection (see Fig. 2(c) where p = 1). 4. SQR-SkipResNet: An innovative variation of the ResNet architecture, where the squared residual is added every other layer. In this approach, the output of each alternate layer is obtained by squaring the previous layer’s output and adding the squared residual to it (see Fig. 2(c)-(d) where p = 2). The numerical experiments were executed on a computer equipped with an Intel(R) Core(TM) i9-9900 CPU operating at 3.10GHz with a total of 64.0 GB of RAM. For the first example, three test functions are investigated and depicted in Fig. 3. The top panel of Fig. 3 displays the 3D surface plot of the test functions, while the bottom panel presents the corresponding contour plots. F1 is a smooth function, originally introduced by Franke [9], which has been extensively used for studying radial basis function (RBF) interpolation. On the other hand, F2 and F3 are non-smooth functions [10]. Example 1 Our observations from these plots are as follows: 1. Plot (a) indicates that the ResNet is not accurate enough compared to the other three networks, both during training and validation. This pattern has been consistently observed in various examples, and we will no longer investigate the ResNet performance. 2. As indicated by the plot, it can be observed that the Plain NN necessitates approximately 2400 iterations for convergence, whereas the proposed SQR-SkipResNet achieves convergence in a significantly reduced 1400 iterations. Additionally, the latter method exhibits higher accuracy compared SQR-SkipResNet. 3. Plot (a) also shows that SkipResNet performs somewhat between Plain NN and SQR-SkipResNet. This behavior has been observed in different examples conducted by the authors, but we do not plan to further investigate to the former. 4. Contour error plots for both Plain NN and SQR-SkipResNet are presented in plots (b) and (c) respectively. These plots highlight that the maximum absolute error achieved with SQR-SkipResNet exhibits a remarkable improvement of approximately 60% compared to Plain NN. Therefore, a higher accuracy and better convergence are observed when using SQRSkipResNet compared to other algorithms. More investigations on the performance of the SQR-SkipResNet has been done by interpolating the non-smooth functions F2 and F3. Figure 6 presents the interpolation results for F2 on the top panel and F3 on the bottom panel with n = 1000. The corresponding training and validation error with respect to the epoch are shown in the Table 1: Example 1: Maximum absolute errors and CPU time (t) for F2-F3. The second and third columns show the interpolated surface using Plain NN and SQR-SkipResNet, respectively. Clearly, a better surface interpolation has been carried using the proposed method. More details are listed in Table 1. This table shows that the accuracy using the SQR-SkipResNet is slightly better than Plain NN, however it is worth nothing that these functions are non-smooth and a slightly changes in error would affect the quality of interpolation tremendously as shown in Fig. 6. However to reach a better accuracy, the SQR-SkipResnet requires larger number of iterations and consequently the higher CPU time. This can be seen as the trade-off that SQRSkipResNet makes for interpolating non-smooth functions to obtain better accuracy, in contrast to the smaller CPU time it requires for interpolating smooth functions. In this example, we demonstrate the performance of the proposed method in a real case study. Specifically, we interpolate the Mt. Eden or Maungawhau volcano in Auckland, NZ, as depicted in Fig. 7(a) [11]. The available data consists of 5307 elevation points uniformly distributed in a mesh grid area of size 10 by 10 meters. Plot (b) shows the 3D surface, and plot (c) presents the contour plot of the volcano. Example 2 Figure 7: Example 2: (a) An image showcasing the Mt. Eden or Maungawhau volcano located in Auckland, New Zealand [11]. (b) A 3D surface representation generated from a dataset containing n = 5307 data points. (c) A contour plot providing insights into the topography of Mt. Eden. Additionally, we provide more details on the interpolated surface and accuracy in Fig. 8. The second row shows the interpolated surface using Plain NN, while the third row shows the results obtained with SQR-SkipResNet. Specifically, plots 8(b) and 8(e) depict the interpolated surfaces for Plain NN and SQR-SkipResNet, respectively. Similarly, plots 8(c) and 8(f) display the contour plots for both methods. Finally, plots 8(d) and 8(g) represent the contour error plots, measured by the maximum absolute error, for Plain NN and SQR-SkipResNet, respectively. Clearly, the results using SQR-SkipResNet significantly outperform those from Plain NN. The accuracy of Plain NN, specifically in terms of the maximum absolute error, improves significantly (500%) when using the SQR-SkipResNet architecture. This underscores the superiority of SQR-SkipResNet in achieving more accurate and reliable interpolation results. In our second experiment, we repeat the the previous example but this time we use Adam optimizer with the learning rate of 1.0E-3, and 10k iteration. The organization of plots are as the previous example. Plot 9(a) shows that SQR-SkipResNet works much more accurate from the beginning of the iterations with much less fluctuation compare with Plain NN (compare plots 9 (b) and 9(e), respectively, and its corresponding contour plots in 9(c) and 9(f)). We also see that the interpolated surface when using the SQRSkipResNet (plot 9(g)) can be completely better than Plain NN (plot 9(d)). The accuracy with respect to the maximum absolute error for the latter one is about 462% better than the Plain NN. A comparisopn between these two optimizers, L-BFGS-B (Fig. 8 and Fig. 9) shows a better performance using Adam for both Plain NN and proposed SQRSkipResNet. Finally we see that in all cases, SQR-SkipResNet led to better accuracy compare to Plain NN. In the concluding example regarding the interpolation problems, we analyze the effectiveness of the proposed neural network in a 3D example, specifically using the Stanford bunny model, as depicted in Fig. 10(a). The entire bunny model has been scaled by a factor of 10. A distribution of points over the bunny’s surface is illustrated in Fig. 10(b), comprising a total of 8171 data points. The validation error is performed using the following test function (refer to [12], F4): Example 3 Figure 11: Example 3: Error profile comparison for the Stanford Bunny model using LBFGS-B (top panel) and Adam optimizers (bottom panel). Training errors are indicated by the dotted line, and validation errors are represented by the solid line. In Fig. 11, the training process (dotted line) is depicted with 500 data values, while the remaining 7671 points are reserved for validation error assessment (solid line). The top panel showcases results obtained using the L-BFGS-B optimizer, while the bottom panel displays outcomes achieved through the Adam optimizer. As demonstrated in Fig. 11(a), the SQR-SkipResNet surpasses the Plain NN in terms of accuracy and convergence rate across both the training and test datasets. The recorded CPU times amount to 35 seconds for Plain NN and 15 seconds for SQR-SkipResNet. Plots (b) and (c) offer insight into the maximum absolute error, highlighting an accuracy improvement of approximately 70% when implementing the proposed network architecture. Moreover, the lower panel of the figure reveals that the efficacy of the SQR-SkipResNet method persists even when utilizing the Adam optimizer. Plot (a) illustrates a more rapid convergence rate for the proposed method when evaluated against test data. The plots (b) and (c) portraying the maximum absolute error clearly exhibit significantly improved accuracy achieved through the proposed approach. This consistent superiority serves to highlight the distinct advantages of the SQR-SkipResNet approach over its alternatives. In comparing the L-BFGS-B and Adam optimizers, it becomes evident that the former displays enhanced performance in both accuracy and CPU time, accomplishing the desired accuracy level more efficiently. To illustrate this aspect, we emphasize the significance of network depth in neural networks, as shown in Fig. 12, specifically focus focusing on F4 with One might wonder about the advantages of employing deep neural networks and their computational implications. = 500 data points and employing the L-BFGS-B optimizer. The results presented here encompass scenarios with 5, 10, and 20 hidden layers, each consisting of 50 neurons. n Conversely, in the case of SQR-SkipResNet, a deeper network correlates with improved convergence rate and enhanced accuracy. This suggests that deeper hidden layers can identify features when embedded within an appropriate neural network architecture for this particular example. This stands in contrast to our findings in the second example (Table 2), which highlighted the problem-dependent nature of selecting an optimal number of layers. In this context, the recorded CPU times for models with 5 and 20 hidden layers amount to 19 and 16 seconds, respectively. This observation suggests that deeper networks may not necessarily result in longer CPU times; rather, they can potentially expedite training due to improved convergence rates, as evident in this case. Comparing the two plots, we observe that the accuracy difference between Plain NN and SQR-SkipResNet becomes more pronounced as the network size increases. This emphasizes the crucial role of architecture selection in achieving stable results.

Physics-Informed with Power-Enhanced Residual Network: Numerical Results

This paper is available on arxiv under CC 4.0 license. Authors: (1) Ulysse Gazin, Universit´e Paris Cit´e and Sorbonne Universit´e, CNRS, Laboratoire de Probabilit´es, Statistique et Mod´elisation, (2) Gilles Blanchard, Universit´e Paris Saclay, Institut Math´ematique d’Orsay, (3) Etienne Roquain, Sorbonne Universit´e and Universit´e Paris Cit´e, CNRS, Laboratoire de Probabilit´es, Statistique et Mod´elisation. Table of Links Abstract & Introduction Main results Application to prediction intervals Application to novelty detection Conclusion, Acknowledgements and References Appendix A: Exact formulas for Pn,m Appendix B: Numerical bounds and templates Appendix C: Proof Appendix D: Explicit control of (16) for α=0 Appendix E: Proof of Corollary 4.1 Appendix F: The Simes inequality Appendix G: Uniform FDP bound for AdaDetect Appendix H:  Additional experiments B Numerical bounds and templates The bound proposed in Theorem 2.3 are explicit and elegant, but can be conservative in some cases and we develop here the numerical approach mentioned in Remark 2.5. Hence, let us consider

Transductive Conformal Inference With Adaptive Scores: Numerical Bounds and Templates

This paper is available on arxiv under CC BY-NC-SA 4.0 DEED license. Authors: (1) Flor Ortiz, University of Luxembour; (2) Juan A. Vasquez-Peralvo, University of Luxembour; (3) Jorge Querol, University of Luxembour; (4) Eva Lagunas, University of Luxembour; (5) Jorge L. Gonzalez Rios, University of Luxembour; (6) Marcele O. K. Mendonc¸a, University of Luxembour; (7) Luis Garces, University of Luxembour; (8) Victor Monzon Baeza, University of Luxembour; (9) Symeon Chatzinotas, University of Luxembou. Table of Links Abstract & Introduction System Model and Problem Statement Antenna Design and Training Data Generation Supervised Learning for Adaptive Beamforming Numerical Results Conclusions, Acknowledgment, and References V. NUMERICAL RESULTS The neural networks are trained offline, so they are used only for onboard satellite inference which drastically decreases the execution time, as explained in [10]. Both approaches are suitable for real-time beamforming adaptation due to the speed where Precision measures the model’s ability not to misclassify negative instances, and Recall measures the model’s ability to find all positive instances. Figure 2 shows the average KPIs obtained for the three approaches: Genetic Algorithm, Multi-Label Classification Neural Network (Approach 1), and Clustering and Classification Neural Network (Approach 2) after running and testing the three approaches iteratively with a sample size greater than 50,000 samples. The Genetic Algorithm outperforms almost all the evaluated KPIs. However, the time required for execution and obtaining the beamforming matrix makes this algorithm unsuitable for real-time applications. On the other hand, both supervised learning-based approaches significantly reduce the execution time by more than a thousand times. This is because after training the ML models, they can be used for inference with almost immediate response times, making them suitable for real-time adaptation. Additionally, both ML-based approaches consistently maintain system performance above 90 Regarding classification metrics, Approach 2 outperforms Approach 1 because binary classification is typically less complex than multi-label classification. However, in terms of overall system performance, Approach 1 performs better than Approach 2.

Supervised Learning Based Real-Time Adaptive Beamforming On-board: Numerical Results

Authors: (1) Davide Viviano, Department of Economics, Harvard University; (2) Lihua Lei, Graduate School of Business, Stanford University; (3) Guido Imbens, Graduate School of Business and Department of Economics, Stanford University; (4) Brian Karrer, FAIR, Meta; (5) Okke Schrijvers, Meta Central Applied Science; (6) Liang Shi, Meta Central Applied Science. Table of Links Abstract & Introduction Setup (When) should you cluster? Choosing the cluster design Empirical illustration and numerical studies Recommendations for practice References A) Notation B) Endogenous peer effects C) Proofs 5 Empirical illustration and numerical studies In this section we illustrate the properties of the procedure in two empirical applications, one using unique data from the Facebook friendship and messaging graphs, and one using data from an experiment conducted in rural China by Cai et al. [2015]. We provide further evidence supporting our theoretical analysis using simulated networks in Section 5.3. 5.1 Clustering on Facebook graphs We first study the procedure’s properties for two clustering algorithms implemented at scale on social networks owned by Meta: Louvain algorithm [Blondel et al., 2008], Balanced Partitioning [Kabiljo et al., 2017]. We consider two graphs owned by Meta. In each graph, edges are continuous variables. In the first graph, edges capture the strength of the friendship in the Facebook graph, and in the second graph they capture connections based on messaging on Facebook, in both cases the data was aggregated and de-identified. We compute our statistics by setting the edges to be zero if below the 5th, 10th, and 50th percentile. We will refer to these graphs obtained as dense, moderate (mod), and sparse after thresholding (graphs all have a bounded maximum degree). We report the bias and variance, with the variance weighted by a parameter ξn, defined respectively as Louvain and Balanced Partitioning clustering produce a hierarchical clustering structure with a growing number of clusters. We consider three “types” of such algorithms, corresponding to three levels in the clustering structure hierarchy (i.e., different numbers of clusters), defined as Type 1, 2, and 3, respectively. For Louvain clustering, higher-order types denote more clusters, whereas it is the opposite for Balanced Partitioning. For each clustering, we report log(Kn/n), log-ratio of the number of clusters over the population size. Table 2 collects each of these algorithms’ worst-case bias and variance and the number of clusters per individual. Louvain algorithm dominates Balanced Partitioning both in worstcase variance and bias. A larger number of clusters increases the bias but decreases the variance, which we observe throughout all graphs. The bias increases for sparser networks because the denominator |Ni | decreases. Figure 3 reports the objective function for different graphs, Louvain algorithms, and degrees of sparsity. For ξn ≈ 1, the Louvain algorithm with the largest number of clusters (Type 3) dominates the other two algorithms in all but one case. This result provides some interpretable comparisons for the number of clusters. We also observe a trade-off between the number of clusters and the degree of sparsity of the adjacency matrix as ξn decreases. Louvain clustering with the largest number of clusters (Type 1) is optimal for a dense graph with ξn ≫ 1. Such clustering, however, is sub-optimal as the graph becomes more sparse. This result is intuitive: for denser networks, a larger number of clusters may best control the estimator’s bias relative to the bias induced by other clustering algorithms. These comparisons motivate trade-offs in the choice of the graph (and its density) and the clustering algorithm. We recommend practitioners compare clustering by averaging objective functions over different sparsity thresholding. For example, it is possible to compare clustering based on the average objective across different graphs using some priors on the degree of sparsity without affecting our theoretical results. Finally, note that using Theorem 3.6 it is possible to compute the smallest values of the spillover effects that would motivate using a cluster instead of Bernoulli design. From Theorem 3.6, researchers should prefer a cluster over a Bernoulli design if In summary, our results shed light on using clustering algorithms for large-scale implementation on online platforms. These results suggest that Louvain clustering with possibly many clusters performs best in practice. It confirms intuitive arguments in Karrer et al. [2021] who also recommend Louvain clustering based on AA-tests. 5.2 Clustering in the field We consider as an application the problem of informing individuals to increase insurance takeup studied in Cai et al. [2015]. The authors collected network information in approximately 184 villages in rural China from 48 larger regions. We use network data collected by Cai et al. [2015] to study the properties of the proposed method, where we assume that two individuals are connected if at least one of the two indicates the other as a friend. Because we do not require information from end-line outcomes, we construct a network using information from surveyed individuals as well as their friends. The network has in total 7649 nodes, once we also include individuals who are friends of surveyed individuals (but do not necessarily live in their same village or were surveyed in the experiment). Individuals between villages present on average 50% of their connections outside their village. On the other hand, individuals in different regions (with approximately 50 regions) have 99% of their connections within the same region. We use an adjacency matrix a “weak ties” adjacency matrix, where two individuals are connected if either indicates the other has a friend. We study clustering within each region and report results for different clustering algorithms. Clustering within each region is performed as in Algorithm 1, where we first estimate Xˆ (K) via semi-definite programming, use K-means algorithm to retrieve the clusters from Xˆ (K) and iterate to estimate the optimal number of clusters K. We report the frontier with respect to the absolute bias consistently with Algorithm 1. 5.2.1 How many clusters should we choose? Figure 4 presents the main results. The left-hand side panel reports the average number of estimated clusters, estimated by the proposed method divided by the population size in a given cluster. Different choices of ξn justify a different number of clusters. For ξn = 1, the number of clusters is 7%, the total population size, whereas for ξn = 15, the number of clusters is roughly half the population size. To gain further intuition of its practical implication, take ψ¯ = 0.24 approximately equal to the outcomes’ variance in Cai et al. [2015] and ϕ¯ n = 0.27 as in Table 2 in Cai et al. [2015]. A recommended choice of ξn is approximately ψ/¯ ϕ¯2 n = 3.29. This choice puts slightly more weight on the bias than the variance when λ = 1, and therefore can be considered a conservative choice for the number of clusters.[13] Even with such a conservative choice, the suggested number of clusters is large relative to the population size. In this case, the optimal number of clusters is around 15% of the overall population size, approximately 600 clusters with a surveyed sample size of 3600 individuals. This number is consistent (in scale) with the choice in the number of clusters in other applications in development, such as Egger et al. [2022], Alatas et al. [2012], but larger than the number of villages that we have in the application in Cai et al. [2015]. 5.2.2 Comparison with other clustering methods Next, we compare the procedure (denoted as “Causal Clustering”) with the following alternative clustering algorithms: ε-net clustering as in Eckles et al. [2017] with ε = 3 as suggested in Eckles et al. [2017]; spectral clustering with a fixed number of clusters equal to n/3, where n is the population size in a given region (note that spectral clustering do not optimize over the number of clusters); Louvain clustering with default parameters selected by the R-package igraph; clustering based on village identity of each individual (creating one additional cluster for those individuals whose village is missing in the data). We contrast the objective function with alternative algorithms in the right-hand side of Figure 4. The proposed method achieves the smallest objective function across all competitors, providing suggestive evidence that Algorithm 1 (with a semi-definite relaxation) works particularly well in practice. For a smaller number of clusters the method is closer to the Louvain clustering algorithm, while for a larger number of clusters the objective of the algorithm is close to the objective of the spectral algorithm. Interestingly, clustering based on village identity leads to the largest loss function. This is because individuals are connected within and between villages in this application. Finally, in Figure 5 (first three panels), we report the mean-squared error obtained by simulating the model in Cai et al. [2015] (Table 2, Column 4) and vary the variance of the residuals σ 2 ∈ {1/4, 1/2, 1} to emulate settings with high, medium and low signal to noise ratio. The model in Cai et al. [2015] assumes that individual outcomes depend on neighbors’ treatments, consistently with our Assumption 1. Our method uniformly outperforms competitors in terms of mean-squared error, except for the Louvain algorithm when the signal-to-noise ratio is particularly high, but not in the remaining settings. In the last panel, we consider settings where the local interference assumption is violated, and we calibrate our model to the specification in Cai et al. [2015] (Column 4, Table 5), where the individual outcome depends on the neighbors’ outcomes, violating Assumption 1 and introducing global dependence. We fix the residuals’ variance to be σ 2 = 1/4, whereas the results are robust as we increase σ 2 . In this setup, we observe a larger mean-squared error of all methods, with the proposed method achieving the lowest mean-squared error. 5.3 A numerical study Next, we illustrate the properties of the method in numerical studies. We consider three different data generating processes for the network formation: a geometric network formation, Albert Barabasi network and Erdos-Renyi graph. The geometric network takes the form Ai,j = 1{|Xi,1 − Xj,1|/2 +|Xi,2 − Xj,2|/2 ≤ rn} where rn = p 4/2.75n similarly to simulations in Leung [2020]. Here, Xi,1, Xi,2 are drawn independently from a uniform distribution between [−1, 1]. For the Albert-Barabasi network we first draw n/5 edges uniformly according to Erdos-Renyi graph with probabilities p = 10/n, and second, we draw sequentially connections of the new nodes to the existing ones with probability equal to the number of connection of each pre-existing node divided by the overall number of connections. Finally, the Erdos-Renyi graph is generated with probability of connection p = 2/n. We study the properties of the proposed method and of the same competitors discussed in Section 5.2. Figure 6 showcases that the proposed procedure leads to the lowest objective function. This result is consistent with our theoretical findings. Figure 7 showcases that the number of clusters increases in the sample size, varies substantially across different data generating processes, and increases in ξn. [13] This follows from the fact that ξn is proportional to ψ/¯ ϕ¯2 n for clusters whose bias is of order O(ϕ¯ n), and researcher’s goal is to minimize the worst-case mean squared error (λ = 1). This paper is under CC 1.0 license. available on arxiv

Design of Cluster Experiments: Empirical Illustration and Numerical Studies

The Story-book of Science by Jean-Henri Fabre, is part of the HackerNoon Books Series. You can jump to any chapter in this book here . A NUMEROUS FAMILY CHAPTER VII. A NUMEROUS FAMILY “APLANT-LOUSE, we will suppose,” resumed Uncle Paul, “has just established itself on the tender shoot of a rosebush. It is alone, all alone. A few days after, young plant-lice surround it: they are its sons. How many are there? Ten, twenty, a hundred? Let us say ten. Is that enough to assure the preservation of the species? Don’t laugh at my question. I know well that if the plant-lice were missing from the rosebushes, the order of things would not be sensibly changed.” “The ants would be the most to be pitied,” said Emile. “The round earth would continue to turn just the same, even when the last plant-louse was dying on its leaf; but it is not, in truth, an idle question to ask if ten plant-lice suffice to preserve the race; for science has no higher object than the quest of providential means for maintaining everything in a just measure of prosperity. “Well, ten plant-lice coming from one would be far too many if we did not have to take account of destructive agencies. One replacing one, the population remains the same; ten replacing one, in a short time the number increases beyond all possible limits. Think of the dervish’s grain of wheat doubled sixty-four times, so that it becomes a bed of wheat of a finger’s depth over the whole earth. What would it be if it had been multiplied ten times instead of doubled! In like manner, after a few years, the descendants of a first plant-louse, continually multiplied tenfold, would be in straitened circumstances in this world. But there is the great reaper, death, which puts an invincible obstacle to overcrowding, counterbalances life in its overgrowing fecundity, and, in partnership with it, keeps all things in a perpetual youth. On a rosebush apparently most peaceful there is death every minute. But the small, the humble, and weak, are the habitual pasture, the daily bread, of the large eaters. To how many dangers is not the plant-louse exposed, so tiny, so weak, and without any means of defense! No sooner does a little bird, hardly out of the shell, discover with its piercing eyes a spot haunted by the plant-lice, than, merely as an appetizer, it will swallow hundreds. And if a worm, far more rapacious, a horrible worm expressly created and put into the world to eat you alive, joins in, ah! my poor plant-lice, may God, the good God of little creatures, protect you; for your race is indeed in peril. “This devourer is of a delicate green with a white stripe on its back. It is tapering in front, swollen at the back. When it doubles itself up it takes the shape of a tear-drop. They call it the ants’ lion because of the ravages it makes in the stupid herd. It establishes itself among them. With its pointed mouth, it seizes one, the biggest, the plumpest; it sucks it and throws away the skin, which is too hard for it. Its pointed head is lowered again, a second plant-louse seized, raised from the leaf, and sucked. Then another and another, a twentieth, a hundredth. The foolish herd, whose ranks are thinning, do not even seem to perceive what is going on. The trapped plant-louse kicks between the lion’s fangs; the others, as if nothing were happening, continue to feed peacefully. It would take a good deal more than that to spoil their appetite! They eat while they are waiting to be eaten. The lion has had enough. He squats amidst the herd to digest at his ease. But digestion is soon over and already the greedy worm has its eye on those that he will soon crunch. After two weeks of continual feasting, after having browsed as it were on whole herds of plant-lice, the worm turns into an elegant little dragon-fly with eyes as bright as gold, and known as the hemerobius. Ladybug (a) larva (b) pupa (c) first joint of larva “Is that all? Oh, no. Here is the lady-bug, the good God’s bug. It is round and red, with black spots. It is very pleasing; it has an innocent air. Who would take it also to be a devourer, filling its stomach with plant-lice? Look at it closely on the rosebush, and you will see it at its ferocious feasting. It is very pretty and innocent-looking; but it is a glutton, there is no denying the fact, so fond is it of plant-lice. “Is that all? Oh, no. Those poor plant-lice are manna, the regular diet of all sorts of ravagers. Young birds eat them, the hemerobius eats them, lady-birds eat them, gluttons of all kinds eat them; and still there are always plant-lice. Ah! that is where, in the fight between fecundity which repairs and the rough battle of life which destroys, the weak excel by opposing legions and legions to the chances of annihilation. In vain the devourers come from all sides and pounce upon their prey; the devoured survive by sacrificing a million to preserve one. The weaker they are, the more fruitful they are. “The herring, cod, and sardine are given over as pasturage for the devourers of the sea, earth, and sky. When they undertake long voyages to graze in favorable spots, their extermination is imminent. The hungry ones of the sea surround the school of fish; the famished ones of the sky hover over their route; those of the earth await them on the shore. Man hastens to lend a strong hand to the killing and to take his share of the sea food. He equips fleets, goes to the fish with naval armies in which all nations are represented; he dries in the sun, salts, smokes, packs. But there is no perceptible diminution in the supply; for him the weak are infinite in number. One cod lays nine million eggs! Where are the devourers that will see the end of such a family?” “Nine million eggs!” exclaimed Emile. “Is that a great many?” “Just to count them, one by one, would take nearly a year of ten working hours each day.” “Whoever counted them had lots of patience,” was Emile’s comment. “They are not counted,” replied Uncle Paul; “they are weighed, which is quickly done; and from the weight the number is deduced. “Like the cod in the sea, the plant-lice are exposed on their rosebushes and alders to numerous chances of destruction. I have told you that they are the daily bread of a multitude of eaters. So, to increase their legions, they have rapid means that are not found in other insects. Instead of laying eggs, very slow in developing, they bring forth living plant-lice, which all, absolutely all, in two weeks have obtained their growth and begin to produce another generation. This is repeated all through the season, that is to say at least half the year, so that the number of generations succeeding one another during this period cannot be less than a dozen. Let us say that one plant-louse produces ten, which is certainly below the actual number. Each of these ten plant-lice borne by the first one bears ten more, making one hundred in all; each of these hundred bears ten, in all one thousand; each of the thousand bears ten, in all ten thousand; and so on, multiplying always by ten, eleven times. Here is the same calculation as the dervish’s grain of wheat, which grew with such astonishing rapidity when they multiplied it by two. For the family of the plant-lice the increase is much more rapid, as the multiplication is made by ten. It is true that the calculation stops at the twelfth instead of going on to the sixty-fourth. No matter, the result would stupefy you; it is equal to a hundred thousand millions. To count a cod’s eggs, one by one, would take nearly a year; to count the descendants of one plant-louse for six months would take ten thousand years! Where are the devourers that would see the end of the miserable louse? Guess how much space these plant-lice would cover, as closely packed as they are on the elder branch.” “Perhaps as large a place as our garden,” suggested Claire. “More than that; the garden is a hundred meters long and the same in width. Well, the family of that one plant-louse would cover a surface ten times larger; that is to say, ten hectares. What do you say to that? Is it not necessary that the young birds, little lady-bugs, and the dragon-fly with the golden eyes should work hard in the extermination of the louse, which if unhindered would in a few years overrun the world? “In spite of the hungry ones which devour them, the plant-lice seriously alarm mankind. Winged plant-lice have been seen flying in clouds thick enough to obscure the daylight. Their black legions went from one canton to another, alighted on the fruit trees, and ravaged them. Ah! when God wishes to try us, the elements are not always unchained. He sends against us in our pride the paltriest of creatures. The invisible mower, the feeble plant-louse, comes, and man is filled with fear; for the good things of the earth are in great peril. “Man, so powerful, can do nothing against these little creatures, invincible in their multitude.” Uncle Paul finished the story of the ants and their cows. Several times since, Emile, Jules, and Claire have talked of the prodigious families of the plant-louse and the cod, but rather lost themselves in the millions and thousand millions. Their uncle was right: his stories interested them much more than Mother Ambroisine’s tales. About HackerNoon Book Series: We bring you the most important technical, scientific, and insightful public domain books. This book is part of the public domain. Jean-Henri Fabre (2018). The Story-book of Science. Urbana, Illinois: Project Gutenberg. Retrieved https://www.gutenberg.org/cache/epub/56795/pg56795-images.html This eBook is for the use of anyone anywhere at no cost and with almost no restrictions whatsoever. You may copy it, give it away or re-use it under the terms of the Project Gutenberg License included with this eBook or online at www.gutenberg.org , located at https://www.gutenberg.org/policy/license.html.

A NUMEROUS FAMILY

: offers the , a tool to help Data Scientists automate their weekly submission pipeline for the Numerai tournament. This CLI configures a in and automatically deploys your model to it. tl;dr Numerai numerai-cli Numerai Prediction Node Amazon Web Services (AWS) . 🚀 This guide describes how I set up my own weekly submission pipeline from scratch, using for free Microsoft Azure and python 🤖 Just give me the code: https://github.com/papaemman/azure-functions-with-python What is Numerai? is a quant hedge founded in San Francisco, CA, in 2015, built on thousands of crowdsourced machine learning models. Every week, Numerai gives away data so that Data Scientists worldwide have free, hedge-fund quality data to develop their Machine Learning (ML) / Deep Learning (DL) models — data they would otherwise not have access to. Numerai At the beginning of each round, competitors can stake (a cryptocurrency issued by Numerai) on the predictions they submit if they want to get paid (or "burnt") based on the predictive accuracy of their models. Staking on a model is fundamentally a type of "bet" competitors take towards the performance of their own model's predictions. NMR tokens Numerai Compute CLI Competing in Numerai is an ongoing, never-ending challenge because competitors need to download the new data every week and submit their predictions based on them. is a framework aiming to help participants automate their weekly submission workflow. Using the , participants can "provision" their cloud infrastructure in and deploy their pre-trained models as a Prediction Node that can Numerai Compute numerai-cli Amazon Web Services (AWS) be triggered by Numerai every weekend download new tournament data run the ML models and upload predictions to Numerai The configures a Numerai Prediction Node in Amazon Web Services (AWS). This solution is architected to cost less than on average, but actual costs may vary. numerai-cli $5/month You need 4 things to use Numerai CLI: , , , and . Docker Python Numerai API Keys AWS API Keys Why not just use Numerai Compute? 1. Cost Last week was tough for the crypto market. carried away the whole crypto market. Currently (May 2022), the NMR token has a price of around . Therefore, saving per month and investing them in buying more NMR to stake in your models is an appealing idea. The massive crash of LUNA 13$ 5$ 2. Learn something new When building something from scratch, you have the opportunity to understand how it works under the hood. Last year, I deep-dived into Microsoft Azure cloud technologies, and I wanted to test if my skills were enough to build a real-world solution. 4. Don't have an AWS account Many professionals prefer to work with other Cloud providers than AWS, such as or Google Cloud Platform. Setting up a new AWS account, adding billing information, and finding your way into the AWS console might seem like a lot of work. *, this guide is for you.* Microsoft Azure If you prefer to use Azure Goal Automate my weekly submission pipeline, for the Numerai Tournament, using Azure and python, with zero cost. Azure offers many services adequate to solve this task, but I decided to use Azure Functions after some research. is a cloud service available on-demand that provides all the continually updated infrastructure and resources needed to run your applications. You focus on the pieces of code that matter most to you, and Functions handles the rest. Functions provides serverless compute for Azure. Azure Functions Going for a is a great way to because you don't need to pay for the infrastructure you don't use. And if you already have the code you want to run, setting up an Azure function is . Also, Azure offers a which is more than enough for this project's scope. serverless architecture save money and time easy free tier, Thus my goal was to create an Azure Function that will: ⏲️ be triggered every Sunday 🔋interact with the Numerai API to download numerai_live_data and upload my predictions 💾 interact with Azure Storage to read and store and data pre-trained ML/DL models numerai_live_data numerai_predictions 📥 send a fun email notification after the submission 😊 allow me to enjoy my weekend MS Azure Architecture Design Technical Implementation Prerequisites Azure Account and VSCode Azure Extension Python 3.8 Azure Functions Core Tools Azure Storage Explorer Numerai API Keys Trained ML/DL models and inference python code Setup 1. Setup all prerequisites Download and install the required software Create your accounts in Azure and Numerai 2. Open VSCode and create a new local Azure function project While you don't have any workspace opened in VSCode, go to the extension, find and select _Create New Project…- S_elect , , and a to create a virtual environment- Select the template, give a name to the function, and set up the timer to run every [* 0 0 * * SUN]. You don't need to set up the function to run in short time intervals. Azure functions support " . Azure FUNCTIONS Directory Language (Python) python interpreter Time Trigger Sunday Execute Function now…" to trigger it manually within VSCode for testing purposes 3. Then, the VSCode will create a with a time-trigger function sample code. automatically Local project ## Azure function project codebase structure
.
├── .venv                     # Python Virtual environment
├── .vscode                   # Configuration options for VSCode
├── host.json                 # Configuration options that affect all functions in a function app instance
├── local.settings.json       # Maintains settings used when running functions locally.These settings aren't used when running in Azure
|
├── AzureFunction_1
│ ├── function.json           # Azure Function Settings
│ ├── __init__.py             # Python Code
│ └── readme.md               # Documentation
|
├── AzureFunction_2
│ ├── function.json           
│ ├── __init__.py             
│ └── readme.md               
|
└── requirements.txt          # Package dependencies (Open VSCode and Press F5 or 4. Run the Function locally Run →Start Debugging) To debug, you must select a storage account for internal use by the Azure Functions runtime: Select a , create a , create a and select a for resources. Keep in mind that this is a . subscription new storage account new resource group, location temporary resource group Go to extension, find and go to . Find the function, right-click, and select Azure FUNCTIONS Local Project Execute Function now… Examine functions' logs Stop the function (ctrl+c) Delete the resource group Important note ⚠️ ”, update the extensionBundle “version”: “[2.*, 4.0.0)” If you get an error “connect ECONNREFUSED 127.0.0.1:9091 to version , in host.json file. 5. Publish the project to Azure Go to extension_,_ find and select Azure FUNCTIONS, Deploy to Function App… Select a and subscription create a new function App Select a and for the new resource runtime stack location This will create a new resource group with an App Service Plan, a Function App, a Storage Account, and an Application Insights service. 6. Run the Function in Azure Go to extension, find , open the related and find the function app. Azure FUNCTIONS subscription Open the function app, find the specific function and select This will run the function to the Azure environment. Execute Function now…. You can see the function logs in the Live metrics tab using the Application Insights resource. 7. Download the function app settings Open VSCode and download remote settings in the Local environment for the function App ( and select Azure Functions: Download Remote Settings) F1 This will update the file with info about App service Plan, Function App, Storage Account, and Application Insights. local.settings.json 8. Setup the Azure Storage account. Open the connect with your Azure account and find the related storage account inside the newly created resource group. Azure Storage Explorer, Create a new Blob container called and upload all your pre-trained ML models there. production-models Create a new Blob container called to store numerai_live_data for every round. live-data Create a new Blob container called to store numerai_predictions for every round. predictions-files 9. Purchase the Twilio SendGrid SaaS offering from Microsoft Azure Marketplace To send a notification email every time the function triggered and submitted predictions, I decided to use the Twilio SendGrid's Email API Purchase the Twilio SendGrid offer (free tier) from Microsoft Azure Marketplace and configure your account on their platform. Get API keys from Twilio SendGrid 10. Configure Function App Having the and hardcoded in the source code is a terrible practice. Azure Functions offers a dedicated Function's App configuration to store environmental variables and secrets. Numerai API Keys Twillio Seng Grid API keys Open the , find the and go to . The complete set of Environment Variables is stored in the Application settings by default. This helps developers define input/output bindings with other resources in Azure. Azure portal Function App service Configuration Create 2 new application settings and store Numeai API Keys and SendGridKeys NumeaiAPIKeys →secret_key=***;public_id=***” SendGridKeyss →*** Open VSCode and re-download remote settings to the local environment ( and ) to get the new Application Settings into the file. F1 Azure Functions: Download Remote Settings local.settings.json 11. Write the Python code Now it's time to write (or copy-paste) the python code responsible for reading the numerai live data & pre-trained ML/DL models and submitting predictions using Numerai API. The input and output configurations (configurations and paths to load and store files) are the only thing you have to modify in your previous python code (running locally) to run it as an Azure Function. Instead of local paths, you should use the newly created Storage Account. For this, you need to use the . Azure Storage Python SDK Go to the file and replace your python code. .py init 12. Run the Function locally and verify that it is working Run the locally ( F5 or Run → Start Debugging), go to A_zure extension,_ find the Function in , and select ` Function Local Project ExecutFunctionon now… Check Function's logs and the connection with the Azure Storage Account Stop the Function (ctrl+c) 13. Redeploy and verify the updated App Go to extension, find , and select Azure FUNCTIONS Deploy to function App… After the deployment completes, find the Function under the respective subscription in VSCode, right-click to function, and select Execute Function now… Check Function's logs in the Metrics tab for this Function. Conclusions Last Sunday, I woke up to this fun email from myself, letting me know that everything went as planned. The Azure Function was triggered on Sunday at 00:00:00 UTC, the Numerai submissions were successful, and I was able to enjoy my day! 🌞 Email Notification for successful submission PS. Check the awesome quote at the end. I added a random quote generator inside the function creating the email notification just to spark things up ✨ Get in touch I hope you liked this guide. If you have any questions or just want to introduce yourself, don't hesitate to reach out! Panagiotis Papaemmanouil 📧 papaemman.pan@gmail.com 💬 https://www.linkedin.com/in/panagiotis-papaemmanouil/ 🌐 https://papaemman.com/ 🔥 Check my under the name Numerai Models PAPAEMMAN https://numer.ai/papaemman1 Also Published here

Nume

anume.com.au

11-50 employees

Since n.d.

COMPANY RANKING

NUME

EVERGREEN INDEX #464

Nume WIKI

Apply to get your Tech Company News Page live on HackerNoon.com today!

Nume

anume.com.au

11-50 employees

Since n.d.

COMPANY RANKING

NUME

EVERGREEN INDEX #464

RELATED COMPANIESRANK

STORIES ABOUT NUME

Nume WIKI

Apply to get your Tech Company News Page live on HackerNoon.com today!