Along with Osteo Striga and other guns, glaives are getting sweeping changes. The melee, projectile, and personal shield hybrid weapon is having its projectile speed increased by 30 percent, projectile damage in PvE increased by 25 percent, but projectile damage in PvP decreased.

Destiny 2’s most overpowered SMG gets big changes alongside glaive updates in season 23

It's been four years since AMD launched RDNA, the successor to the venerable GCN graphics architecture. We take a look through the tech and numbers to

4 Years of AMD RDNA: Another Zen or a New Bulldozer?

New partnership will combine transport refrigeration unit and trailer data within one platform leveraging SCALAR fleet orchestration technology.

Thermo King, ZF advance reefer connectivity

Bungie has popped open a rare Engram full of Destiny 2 update 3.04 patch notes for your viewing pleasure, which comes with support for Festival of the Lost (that’s Destiny’s annual Halloween-themed event, in case you didn’t know) as well as various weapon tuning and balancing.

Destiny 2 Update 3.04 Patch Notes Revealed For Festival Of The Lost, Weapon Tuning & More

Grenade energy is now provided on ability kills instead of damage with abilities. Reduced the ADS Damage Falloff Scalar from 1.7x to 1.6x. Increased base damage from 14.25 to 15. Decreased critical hit multiplier from 1.6 to 1.55 (crit damage goes from 22. ...

Destiny 2 Update 7.2.5 patch notes

Mental health problems stemming from lifestyle concerns like stress, loneliness/isolation, anxiety, trauma, debt, bereavement, medical issues, social media and more at an all-time high. In fact, new Gallup findings indicate stress,

3 BioWellness Advancements Stemming Toxic Stress

Get the most recent info and news about Scalar on HackerNoon, where 10k+ technologists publish stories for 4M+ monthly readers.

The World's Reliable Database - 高い信頼性を有する独自のデータベースを活用してデジタライゼーションの加速を支援します

Scalar

Let be a cyclic group, is the element in the group, is the scalar, and the Multi-Scalar Multiplication (MSM) is the algorithm for calculating , the sum of multiple scalar multiplications. As the group operation is more complicated than the addition and multiplication of elements in the finite field, the MSM algorithm is employed to reduce the number of times of group operation as much as possible. Usually, is a cyclic group defined on the elliptic curve of on the finite field of , with the order being bits, and . If the group operation is conducted by the plain algorithm of fast exponentiation, the number of group operations needed for every is times on average, and for all is times in total. Next, we will introduce some optimization methods corresponding to larger . G P_i ∈ G a_i ∈ Z ∑_{i=1}^{n} a_{i} P_{i} G y^2=x^3+ax+b F_p | | G b ≈ 256 0 ≤ a_i< 2^b a_i.P_i 1.5bn a_i.P_i 1.5b n Optimization Based on the Windowing Technique We can split the bit scalar into c width windows: reduce the scalar to base system. b 2^c Here, } , so we reduce the original algorithm issue of -bit MSM into a smaller -bit issue. We can put the same scalars from into a bucket and write them in another form: Q_j=∑_{i=1}^{n a_{i j} P_{i} b c ∑_{i=1}^{n} a_{i j} P_{i} For example, when , it is calculated as: c = 3 When calculating , set the partial sum T_l=S_l+⋯+S_k, then ∑kS_k However, the operation results of can be obtained through the recurrence sequence . It means that only one group operation is needed for the operation for each . Under the MSM algorithm with cc-bit scalar, all requires times of group operation. All requires times and the sum of these requires times of group operation. Therefore, each window with the value of c requires times of group operations. The operation of } only requires the normal method of fast exponentiation by times of group operation. T_l T_l=T_l+1+S_l T_l S_k n−2^c+1 T_k 2^ 1 c− T_k ^c−2 2 n+2^c−2 ∑_{j=0}^{b / c-1} Q_{j} 2^{j c (b/c−1)(c+1)=b − c + b/c−1 In summary, we can conclude that the total number of group operations required by the method of the Windowing Technique of a width of is c Set , then the number of group operations is . When , the number of group operations reduces to of its original value. c = logn 2bn/logn n≈10^5 1/12 Optimization Based on Group Endomorphism For the cyclic group on the elliptic curve on the finite field F_p, if the following group endomorphism of φ can be found: there exists , satisfying holds for all the points on , then it is easy to prove that such an endomorphism is a multiplication map, namely there is a making hold for all the points of on , which means that whenever we know the coordinates of one point, we can change it to the coordinates of another point simply by multiplying a number in to both the values of abscissa and ordinate. The algorithm can be further optimized based on this important property. G y^2=x^3+ax+b α,β ∈ F_p φ(x,y)=(αx,βy) G λ φ(P)=λP P G F_p When . The reverse value of one point can be obtained only by taking the opposite number from the ordinate. Based on this feature, in the plain fast exponentiation algorithm, the scalar can be written into the non-adjacent form (NAF), namely and any two adjacent cannot be non-zero at the same time. Compared to the -bit scalars, the group operation number in the fast exponentiation algorithm can be reduced from the original average of times to times. This technique can also be used to optimize the Windowing Technique. After writing to λ = −1, α = 1, β = −1 e_{i}.2^{i}, e_{i} ∈ {−1,0,1} ∑ e_i b 3/2⋅b 4/3⋅b a_i a_i=∑_{j=0}^{b / c-1} a_{i, j} 2^{j c}, 0 <= a_{i, j}<2^{c} conduct the following operation: The result of and - can be obtained. Since the complexity of addition and subtraction in group operation of an elliptic curve is the same, we can put these terms into a bucket according to the absolute value of scalars. Therefore, the number of groups can be reduced from the original 2c to , and the overall number of group operations required is reduced from (1) to: a_i=∑_{j=0}^{b / c-1} a’_{i, j}^2^{j c} 2^{c-1} <= a’_{i, j}^ <= 2^{c-1} 2^c If the parameters of the elliptic curve are special, for example, the BLS curve can be written as , and , take third-order element α from F_p^x, then there is a corresponding that holds y^2=x^3+b p ≡ 1( mod3 ) λ λ( )=( ) x,y αx,y Algorithm 1: Signed Digits In addition, . Then the multiplication operation can be optimized as follows: λ3 ≡ 1(mod |G| ) From [1], we can find a small enough scalar | |, | |<√ to make the above equation true, which gives us a way to reduce the multiplication of bb bits into the multiplication of two bits. Use it into the Windowing Technique, and the number of group operations is reduced to m1 m2 G b/2 In summary, both the above two optimization methods can reduce the number of group operations by when . 5.5% n=10^5 References [1] Francesco Sica, Mathieu Ciet, and Jean-Jacques Quisquater. Analysis of the gallant-lambert-vanstone method based on eﬀicient endomorphisms: Elliptic and hyperelliptic curves. In International Workshop on Selected Areas in Cryptography, pages 21–36. Springer, 2002.

Optimization of Multi-Scalar Multiplication Algorithm: Sin7Y Tech Review (21)

This paper is available on arxiv under CC 4.0 license.
Authors:
(1) Zheyu Oliver Wang, Department of Aeronautics and Astronautics, Massachusetts Institute of Technology, Cambridge, MA and olivrw@mit.edu;
(2)  Ricardo Baptista, Computing + Mathematical Sciences, California Institute of Technology, Pasadena, CA and rsb@caltech.edu;
(3) Youssef Marzouk, Department of Aeronautics and Astronautics, Massachusetts Institute of Technology, Cambridge, MA and ymarz@mit.edu;
(4) Lars Ruthotto, Department of Mathematics, Emory University, Atlanta, GA and lruthotto@emory.edu;
(5) Deepanshu Verma, Department of Mathematics, Emory University, Atlanta, GA and deepanshu.verma@emory.edu. Table of Links Abstract & Introduction
Background and Related Work
Partially Convex Potential Maps (PCP-Map) for Conditional OT
Conditional OT flow (COT-Flow)
Implementation and Experimental Setup
Numerical Experiments
Discussion and Reference 4. Conditional OT flow (COT-Flow). In this section, we extend the OT-regularized continuous normalizing flows in [36] to the conditional generative modeling problem introduced in section 2. Training Problem. Following the general approach of continuous normalizing flows [11, 19], we express the transport map that pushes forward the reference distribution ρz to the target π(x|y) via the flow map of an ODE. That is, we define g(z, y) = u(1) as the terminal state of the ODE where the evolution of u: [0, 1] → R n depends on the velocity field v : [0, 1] × R n × R m → R n and will be parameterized with a neural network below. When the velocity is trained to minimize the negative log-likelihood loss in (3.2), the resulting mapping is called a continuous normalizing flow [19]. We add y as an additional input to the velocity field to enable conditional sampling. Consequently, the resulting flow map and generator g depend on y. One advantage of defining the generator through an ODE is that the loss function can be evaluated efficiently for a wide range of velocity functions. Recall that the loss function requires the inverse of the generator and the log-determinant of its Jacobian. For sufficiently regular velocity fields, the inverse of the generator can be obtained by integrating backward in time. To be precise, we define g −1 (x, y) = p(0) where p : [0, 1] → R n satisfies (4.1) with the terminal condition p(1) = x. As derived in [58, 19, 56], constructing the generator through an ODE also simplifies computing the log-determinant of the Jacobian, i.e., Penalizing transport costs during training leads to theoretical and numerical advantages; see, for example, [58, 56, 36, 17]. Hence, we consider the OT-regularized training problem where α1 > 0 is a regularization parameter that trades off matching the distributions (for α1 ≪ 1) and minimizing the transport costs (for α1 ≫ 0) given by the dynamic transport cost penalty This penalty is stronger than the static counterpart in (3.4), in other words However, the values of both penalties agree when the velocity field in (4.1) is constant along the trajectories, which is the case for the optimal transport map. Hence, we expect the solution of the dynamic problem to be close to that of the static formulation when α1 is chosen well. To provide additional theoretical insight and motivate our numerical method, we note that (4.3) is related to the potential mean field game (MFG) Here, the terminal and running costs in the objective functional are the L2 anti-derivatives of JNLL and PDOT, respectively, and the continuity equation is used to represent the density evolution under the velocity v. To be precise, (4.3) can be seen as a discretization of (4.6) in the Lagrangian coordinates defined by the reversed ODE (4.14). Note that both formulations differ in the way the transport costs are measured: (4.4) computes the cost of pushing the conditional distribution to the Gaussian, while (4.6) penalizes the costs of pushing the Gaussian to the conditional. While these two terms do not generally agree, they coincide for the L2 optimal transport map; see [16, Corollary 2.5.13] for a proof for unconditional optimal transport. For more insights into MFGs and their relation to optimal transport and generative modeling, we refer to our prior work [43] and the more recent work [57, Sec 3.4]. The fact that the solutions of the microscopic version in (4.3) and the macroscopic version in (4.6) agree is remarkable and was first shown in the seminal work [28]. By Pontryagin’s maximum principle, the solution of (4.3) satisfies the feedback form where Φ : [0, 1] × R n × R m → R and Φ(·, ·, y) can be seen as the value function of the MFG and, alternatively, as the Lagrange multiplier in (4.6) for a fixed y. Therefore, we model the velocity as a conservative vector field as also proposed in [36, 43]. This also simplifies the computation of the log-determinant since the Jacobian of v is symmetric, and we note that Another consequence of optimal control theory is that the value function satisfies the Hamilton Jacobi Bellman (HJB) equations with the terminal condition which is only tractable if the joint density, π(x, y), is available. These n-dimensional PDEs are parameterized by y ∈ R m. Neural network approach. We parameterize the value function, Φ, with a scalar-valued neural network. In contrast to the static approach in the previous section, the choice of function approximator is more flexible in the dynamic approach. The approach can be effective as long as Φ is parameterized with any function approximation tool that is effective in high dimensions, allows efficient evaluation of its gradient and Laplacian, and the training problem can be solved sufficiently well. For our numerical experiments, we use the architecture considered in [36] and model Φ as the sum of a simple feed-forward neural network and a quadratic term. That is, As in [36], we model the neural network as a two-layer residual network of width w that reads Sample generation. After training, we draw i.i.d. samples from the approximated conditional distribution for a given y by sampling z ∼ ρz and solving the ODE (4.1). Since we train the inverse generator using a fixed number of integration steps, nt, it is interesting to investigate how close our solution is to the continuous problem. One indicator is to compare the samples obtained with different number of integration steps during sampling. Another indicator is to compute the variance of ∇Φ along the trajectories. Hyperparameters. As described in section 5, we randomly sample the width, w, of the neural network, the number of time integration steps during training, the penalty parameters α1, α2, and the hyperparameters of the Adam optimizer (batch size and initial learning rate) from the values listed in Table 2. This paper is available on arxiv under CC 4.0 license. Authors: (1) Zheyu Oliver Wang, Department of Aeronautics and Astronautics, Massachusetts Institute of Technology, Cambridge, MA and olivrw@mit.edu; (2)  Ricardo Baptista, Computing + Mathematical Sciences, California Institute of Technology, Pasadena, CA and rsb@caltech.edu; (3) Youssef Marzouk, Department of Aeronautics and Astronautics, Massachusetts Institute of Technology, Cambridge, MA and ymarz@mit.edu; (4) Lars Ruthotto, Department of Mathematics, Emory University, Atlanta, GA and lruthotto@emory.edu; (5) Deepanshu Verma, Department of Mathematics, Emory University, Atlanta, GA and deepanshu.verma@emory.edu. This paper is available on arxiv under CC 4.0 license. Authors: Authors: (1) Zheyu Oliver Wang, Department of Aeronautics and Astronautics, Massachusetts Institute of Technology, Cambridge, MA and olivrw@mit.edu; (2)  Ricardo Baptista, Computing + Mathematical Sciences, California Institute of Technology, Pasadena, CA and rsb@caltech.edu; (3) Youssef Marzouk, Department of Aeronautics and Astronautics, Massachusetts Institute of Technology, Cambridge, MA and ymarz@mit.edu; (4) Lars Ruthotto, Department of Mathematics, Emory University, Atlanta, GA and lruthotto@emory.edu; (5) Deepanshu Verma, Department of Mathematics, Emory University, Atlanta, GA and deepanshu.verma@emory.edu. Table of Links Abstract & Introduction Background and Related Work Partially Convex Potential Maps (PCP-Map) for Conditional OT Conditional OT flow (COT-Flow) Implementation and Experimental Setup Numerical Experiments Discussion and Reference Abstract & Introduction Abstract & Introduction Background and Related Work Background and Related Work Partially Convex Potential Maps (PCP-Map) for Conditional OT Partially Convex Potential Maps (PCP-Map) for Conditional OT Conditional OT flow (COT-Flow) Conditional OT flow (COT-Flow) Implementation and Experimental Setup Implementation and Experimental Setup Numerical Experiments Numerical Experiments Discussion and Reference Discussion and Reference 4. Conditional OT flow (COT-Flow). In this section, we extend the OT-regularized continuous normalizing flows in [36] to the conditional generative modeling problem introduced in section 2. Training Problem. Following the general approach of continuous normalizing flows [11, 19], we express the transport map that pushes forward the reference distribution ρz to the target π(x|y) via the flow map of an ODE. That is, we define g(z, y) = u(1) as the terminal state of the ODE where the evolution of u: [0, 1] → R n depends on the velocity field v : [0, 1] × R n × R m → R n and will be parameterized with a neural network below. When the velocity is trained to minimize the negative log-likelihood loss in (3.2), the resulting mapping is called a continuous normalizing flow [19]. We add y as an additional input to the velocity field to enable conditional sampling. Consequently, the resulting flow map and generator g depend on y. One advantage of defining the generator through an ODE is that the loss function can be evaluated efficiently for a wide range of velocity functions. Recall that the loss function requires the inverse of the generator and the log-determinant of its Jacobian. For sufficiently regular velocity fields, the inverse of the generator can be obtained by integrating backward in time. To be precise, we define g −1 (x, y) = p(0) where p : [0, 1] → R n satisfies (4.1) with the terminal condition p(1) = x. As derived in [58, 19, 56], constructing the generator through an ODE also simplifies computing the log-determinant of the Jacobian, i.e., Penalizing transport costs during training leads to theoretical and numerical advantages; see, for example, [58, 56, 36, 17]. Hence, we consider the OT-regularized training problem where α1 > 0 is a regularization parameter that trades off matching the distributions (for α1 ≪ 1) and minimizing the transport costs (for α1 ≫ 0) given by the dynamic transport cost penalty This penalty is stronger than the static counterpart in (3.4), in other words However, the values of both penalties agree when the velocity field in (4.1) is constant along the trajectories, which is the case for the optimal transport map. Hence, we expect the solution of the dynamic problem to be close to that of the static formulation when α1 is chosen well. To provide additional theoretical insight and motivate our numerical method, we note that (4.3) is related to the potential mean field game (MFG) Here, the terminal and running costs in the objective functional are the L2 anti-derivatives of JNLL and PDOT, respectively, and the continuity equation is used to represent the density evolution under the velocity v. To be precise, (4.3) can be seen as a discretization of (4.6) in the Lagrangian coordinates defined by the reversed ODE (4.14). Note that both formulations differ in the way the transport costs are measured: (4.4) computes the cost of pushing the conditional distribution to the Gaussian, while (4.6) penalizes the costs of pushing the Gaussian to the conditional. While these two terms do not generally agree, they coincide for the L2 optimal transport map; see [16, Corollary 2.5.13] for a proof for unconditional optimal transport. For more insights into MFGs and their relation to optimal transport and generative modeling, we refer to our prior work [43] and the more recent work [57, Sec 3.4]. The fact that the solutions of the microscopic version in (4.3) and the macroscopic version in (4.6) agree is remarkable and was first shown in the seminal work [28]. By Pontryagin’s maximum principle, the solution of (4.3) satisfies the feedback form where Φ : [0, 1] × R n × R m → R and Φ(·, ·, y) can be seen as the value function of the MFG and, alternatively, as the Lagrange multiplier in (4.6) for a fixed y. Therefore, we model the velocity as a conservative vector field as also proposed in [36, 43]. This also simplifies the computation of the log-determinant since the Jacobian of v is symmetric, and we note that Another consequence of optimal control theory is that the value function satisfies the Hamilton Jacobi Bellman (HJB) equations with the terminal condition which is only tractable if the joint density, π(x, y), is available. These n-dimensional PDEs are parameterized by y ∈ R m. Neural network approach. We parameterize the value function, Φ, with a scalar-valued neural network. In contrast to the static approach in the previous section, the choice of function approximator is more flexible in the dynamic approach. The approach can be effective as long as Φ is parameterized with any function approximation tool that is effective in high dimensions, allows efficient evaluation of its gradient and Laplacian, and the training problem can be solved sufficiently well. For our numerical experiments, we use the architecture considered in [36] and model Φ as the sum of a simple feed-forward neural network and a quadratic term. That is, As in [36], we model the neural network as a two-layer residual network of width w that reads Sample generation. After training, we draw i.i.d. samples from the approximated conditional distribution for a given y by sampling z ∼ ρz and solving the ODE (4.1). Since we train the inverse generator using a fixed number of integration steps, nt, it is interesting to investigate how close our solution is to the continuous problem. One indicator is to compare the samples obtained with different number of integration steps during sampling. Another indicator is to compute the variance of ∇Φ along the trajectories. Hyperparameters. As described in section 5, we randomly sample the width, w, of the neural network, the number of time integration steps during training, the penalty parameters α1, α2, and the hyperparameters of the Adam optimizer (batch size and initial learning rate) from the values listed in Table 2.

Efficient Neural Network Approaches for Conditional Optimal Transport:Conditional OT flow (COT-Flow)

This paper is available on arxiv under CC 4.0 license.
Authors:
(1) Zheyu Oliver Wang, Department of Aeronautics and Astronautics, Massachusetts Institute of Technology, Cambridge, MA and olivrw@mit.edu;
(2)  Ricardo Baptista, Computing + Mathematical Sciences, California Institute of Technology, Pasadena, CA and rsb@caltech.edu;
(3) Youssef Marzouk, Department of Aeronautics and Astronautics, Massachusetts Institute of Technology, Cambridge, MA and ymarz@mit.edu;
(4) Lars Ruthotto, Department of Mathematics, Emory University, Atlanta, GA and lruthotto@emory.edu;
(5) Deepanshu Verma, Department of Mathematics, Emory University, Atlanta, GA and deepanshu.verma@emory.edu. Table of Links Abstract & Introduction
Background and Related Work
Partially Convex Potential Maps (PCP-Map) for Conditional OT
Conditional OT flow (COT-Flow)
Implementation and Experimental Setup
Numerical Experiments
Discussion and Reference 3. Partially Convex Potential Maps (PCP-Map) for Conditional OT. Our first approach, PCP-Map, approximately solves the static COT problem through maximum likelihood training over a set of partially monotone maps. Motivated by the structure of transport maps that are optimal with respect to the quadratic cost function, we parameterize these maps as the gradients of scalar-valued neural networks that are strictly convex in x for any choice of the weights. Training problem. Given samples from the joint distribution, our algorithm seeks a conditional generator g by solving the maximum likelihood problem The objective JNLL is the expected negative log-likelihood functional which agrees up to an additive constant with the negative logarithm of (2.1), and M is the set of maps that are monotonically increasing in their first argument, i.e., We note that JNLL also agrees (up to an additive constant) with the Kullback-Leibler (KL) divergence from the push forward of ρz through the generator g to the conditional distribution in expectation over the conditioning variable. Since the learning problem in (3.2) only involves the inverse generator g −1, we seek this map directly in the same space of monotone functions M in (3.3). This avoids inverting the generator during training. The drawback is that sampling the target conditional requires inverting the learned map. In this work, we will find the conditional generator g −1 (·, y) that pushes forward the conditional distribution π(x|y) to the reference distribution for each y. Limiting the search to monotone maps is motivated by the celebrated Brenier’s theorem, which ensures there exists a unique monotone map g −1 such that g −1 (·, y)♯π(x|y) = g(·, y) ♯π(x|y) = ρz among all maps written as the gradient of a convex potential; see [8] for the original result and [10] for conditional transport maps. Theorem 2.3 in [10] also shows that g −1 is optimal in the sense that among all maps that match the distributions, it minimizes the integrated L2 transport costs Neural Network Representation. In this work, we leverage the structural form of the conditional Brenier map by using partially input convex neural networks (PICNNs) [2] to express the inverse generator directly. In particular, we parameterize g −1 as the gradient of a PICNN G˜ θ : R n×R m → R that depends on weights θ. A PICNN is a feed-forward neural network that is specifically designed to ensure convexity in some of its inputs; see the original work that introduced this neural network architecture in [2] and its use for generative modeling in [20]. To the best of our knowledge, investigating if PICNNs are universal approximators of partially input convex functions is still an open issue, also see [23, Section IV], that is beyond the scope of our paper; a perhaps related result for fully input convex neural networks is given in [20, Appendix C]. To ensure the monotonicity of g −1, we construct G˜ to be strictly convex as a linear combination of a PICNN and a positive definite quadratic term. That is, where γ1, γ2, and γ3 are scalar parameters that are re-parameterized via the soft-plus function ψ(x) = log(1 + exp(x)) and the ReLU function σReLU(x) = max{0, x} to ensure strict convexity of G˜ θ. Here, wK is the output of an K-layer PICNN and is computed through forward propagation through the layers k = 0, . . . , K − 1 starting with the inputs v0 = y and w0 = x Using properties for the composition of convex functions [18], it can be verified that the forward propagation in (3.6) defines a function that is convex in x, but not necessarily in y (which is not needed), as long as σ (w) is convex and non-decreasing. To compute the log determinant of g −1, we use vectorized automatic differentiation to obtain the Hessian of G˜ θ with respect to its first input and then compute its eigenvalues. This is feasible when the Hessian is moderate; e.g., in our experiments, it is less than one hundred. We use efficient implementations of these methods that parallelize the computations over all the samples in a batch. Our algorithm enforces the non-negativity constraint by projecting the parameters into the non-negative orthant after each optimization step using ReLU. Thereby, we alleviate the need for re- parameterization, for example, using the softplus function in [2]. Another novelty introduced in PICNN is that we utilize trainable affine layer parameters L (v) k and a context feature width u as a hyperparameter to increase the expressiveness of the conditioning variables, which are pivotal to characterizing conditional distributions; existing works such as [20] set L (v) 1:K−2 = I. Sample generation. Due to our neural network parameterization, there is generally no closedform relation between g and g −1 = G˜ θ. As in [20], we approximate the inverse of G˜ θ during sampling as the Legendre-Fenchel dual. That is, we solve the convex optimization problem Due to the strict convexity of G˜ θ in its first argument, the first-order optimality conditions gives Hyperparameters. In our numerical experiments, we vary only three hyperparameters to adjust the complexity of the architecture. As described in section 5 we randomly sample the depth, K, the feature width, w, and the context width, u, from the values in Table 2. This paper is available on arxiv under CC 4.0 license. Authors: (1) Zheyu Oliver Wang, Department of Aeronautics and Astronautics, Massachusetts Institute of Technology, Cambridge, MA and olivrw@mit.edu; (2)  Ricardo Baptista, Computing + Mathematical Sciences, California Institute of Technology, Pasadena, CA and rsb@caltech.edu; (3) Youssef Marzouk, Department of Aeronautics and Astronautics, Massachusetts Institute of Technology, Cambridge, MA and ymarz@mit.edu; (4) Lars Ruthotto, Department of Mathematics, Emory University, Atlanta, GA and lruthotto@emory.edu; (5) Deepanshu Verma, Department of Mathematics, Emory University, Atlanta, GA and deepanshu.verma@emory.edu. This paper is available on arxiv under CC 4.0 license. Authors: Authors: (1) Zheyu Oliver Wang, Department of Aeronautics and Astronautics, Massachusetts Institute of Technology, Cambridge, MA and olivrw@mit.edu; (2)  Ricardo Baptista, Computing + Mathematical Sciences, California Institute of Technology, Pasadena, CA and rsb@caltech.edu; (3) Youssef Marzouk, Department of Aeronautics and Astronautics, Massachusetts Institute of Technology, Cambridge, MA and ymarz@mit.edu; (4) Lars Ruthotto, Department of Mathematics, Emory University, Atlanta, GA and lruthotto@emory.edu; (5) Deepanshu Verma, Department of Mathematics, Emory University, Atlanta, GA and deepanshu.verma@emory.edu. Table of Links Abstract & Introduction Background and Related Work Partially Convex Potential Maps (PCP-Map) for Conditional OT Conditional OT flow (COT-Flow) Implementation and Experimental Setup Numerical Experiments Discussion and Reference Abstract & Introduction Abstract & Introduction Background and Related Work Background and Related Work Partially Convex Potential Maps (PCP-Map) for Conditional OT Partially Convex Potential Maps (PCP-Map) for Conditional OT Conditional OT flow (COT-Flow) Conditional OT flow (COT-Flow) Implementation and Experimental Setup Implementation and Experimental Setup Numerical Experiments Numerical Experiments Discussion and Reference Discussion and Reference 3. Partially Convex Potential Maps (PCP-Map) for Conditional OT. Our first approach, PCP-Map, approximately solves the static COT problem through maximum likelihood training over a set of partially monotone maps. Motivated by the structure of transport maps that are optimal with respect to the quadratic cost function, we parameterize these maps as the gradients of scalar-valued neural networks that are strictly convex in x for any choice of the weights. Training problem. Given samples from the joint distribution, our algorithm seeks a conditional generator g by solving the maximum likelihood problem The objective JNLL is the expected negative log-likelihood functional which agrees up to an additive constant with the negative logarithm of (2.1), and M is the set of maps that are monotonically increasing in their first argument, i.e., We note that JNLL also agrees (up to an additive constant) with the Kullback-Leibler (KL) divergence from the push forward of ρz through the generator g to the conditional distribution in expectation over the conditioning variable. Since the learning problem in (3.2) only involves the inverse generator g −1, we seek this map directly in the same space of monotone functions M in (3.3). This avoids inverting the generator during training. The drawback is that sampling the target conditional requires inverting the learned map. In this work, we will find the conditional generator g −1 (·, y) that pushes forward the conditional distribution π(x|y) to the reference distribution for each y. Limiting the search to monotone maps is motivated by the celebrated Brenier’s theorem, which ensures there exists a unique monotone map g −1 such that g −1 (·, y)♯π(x|y) = g(·, y) ♯π(x|y) = ρz among all maps written as the gradient of a convex potential; see [8] for the original result and [10] for conditional transport maps. Theorem 2.3 in [10] also shows that g −1 is optimal in the sense that among all maps that match the distributions, it minimizes the integrated L2 transport costs Neural Network Representation. In this work, we leverage the structural form of the conditional Brenier map by using partially input convex neural networks (PICNNs) [2] to express the inverse generator directly. In particular, we parameterize g −1 as the gradient of a PICNN G˜ θ : R n×R m → R that depends on weights θ. A PICNN is a feed-forward neural network that is specifically designed to ensure convexity in some of its inputs; see the original work that introduced this neural network architecture in [2] and its use for generative modeling in [20]. To the best of our knowledge, investigating if PICNNs are universal approximators of partially input convex functions is still an open issue, also see [23, Section IV], that is beyond the scope of our paper; a perhaps related result for fully input convex neural networks is given in [20, Appendix C]. To ensure the monotonicity of g −1, we construct G˜ to be strictly convex as a linear combination of a PICNN and a positive definite quadratic term. That is, where γ1, γ2, and γ3 are scalar parameters that are re-parameterized via the soft-plus function ψ(x) = log(1 + exp(x)) and the ReLU function σReLU(x) = max{0, x} to ensure strict convexity of G˜ θ. Here, wK is the output of an K-layer PICNN and is computed through forward propagation through the layers k = 0, . . . , K − 1 starting with the inputs v0 = y and w0 = x v0 = y w0 = x Using properties for the composition of convex functions [18], it can be verified that the forward propagation in (3.6) defines a function that is convex in x, but not necessarily in y (which is not needed), as long as σ (w) is convex and non-decreasing. To compute the log determinant of g −1, we use vectorized automatic differentiation to obtain the Hessian of G˜ θ with respect to its first input and then compute its eigenvalues. This is feasible when the Hessian is moderate; e.g., in our experiments, it is less than one hundred. We use efficient implementations of these methods that parallelize the computations over all the samples in a batch. Our algorithm enforces the non-negativity constraint by projecting the parameters into the non-negative orthant after each optimization step using ReLU. Thereby, we alleviate the need for re- parameterization, for example, using the softplus function in [2]. Another novelty introduced in PICNN is that we utilize trainable affine layer parameters L (v) k and a context feature width u as a hyperparameter to increase the expressiveness of the conditioning variables, which are pivotal to characterizing conditional distributions; existing works such as [20] set L (v) 1:K−2 = I. Sample generation. Due to our neural network parameterization, there is generally no closedform relation between g and g −1 = G˜ θ. As in [20], we approximate the inverse of G˜ θ during sampling as the Legendre-Fenchel dual. That is, we solve the convex optimization problem Due to the strict convexity of G˜ θ in its first argument, the first-order optimality conditions gives Hyperparameters. In our numerical experiments, we vary only three hyperparameters to adjust the complexity of the architecture. As described in section 5 we randomly sample the depth, K, the feature width, w, and the context width, u, from the values in Table 2.

Efficient Neural Network Approaches: Partially Convex Potential Maps (PCP-Map) for Conditional OT

This paper is available on arxiv under CC 4.0 license.
Authors:
(1) Zheyu Oliver Wang, Department of Aeronautics and Astronautics, Massachusetts Institute of Technology, Cambridge, MA and olivrw@mit.edu;
(2)  Ricardo Baptista, Computing + Mathematical Sciences, California Institute of Technology, Pasadena, CA and rsb@caltech.edu;
(3) Youssef Marzouk, Department of Aeronautics and Astronautics, Massachusetts Institute of Technology, Cambridge, MA and ymarz@mit.edu;
(4) Lars Ruthotto, Department of Mathematics, Emory University, Atlanta, GA and lruthotto@emory.edu;
(5) Deepanshu Verma, Department of Mathematics, Emory University, Atlanta, GA and deepanshu.verma@emory.edu. Table of Links Abstract & Introduction
Background and Related Work
Partially Convex Potential Maps (PCP-Map) for Conditional OT
Conditional OT flow (COT-Flow)
Implementation and Experimental Setup
Numerical Experiments
Discussion and ReferenceBayesianInference Abstract. We present two neural network approaches that approximate the solutions of static and dynamic conditional optimal transport (COT) problems, respectively. Both approaches enable sampling and density estimation of conditional probability distributions, which are core tasks in Bayesian inference. Our methods represent the target conditional distributions as transformations of a tractable reference distribution and, therefore, fall into the framework of measure transport. COT maps are a canonical choice within this framework, with desirable properties such as uniqueness and monotonicity. However, the associated COT problems are computationally challenging, even in moderate dimensions. To improve the scalability, our numerical algorithms leverage neural networks to parameterize COT maps. Our methods exploit the structure of the static and dynamic formulations of the COT problem. PCP-Map models conditional transport maps as the gradient of a partially input convex neural network (PICNN) and uses a novel numerical implementation to increase computational efficiency compared to state-of-the-art alternatives. COT-Flow models conditional transports via the flow of a regularized neural ODE; it is slower to train but offers faster sampling. We demonstrate their effectiveness and efficiency by comparing them with state-of-the art approaches using benchmark datasets and Bayesian inverse problems. Key words. Measure transport, generative modeling, optimal transport, Bayesian inference, inverse problems, uncertainty quantification MSC codes. 62F15, 62M45 Most effective sampling techniques are limited to the conventional Bayesian setting as they require a tractable likelihood model (often given by a forward operator that maps x to y and a noise model) and prior. Even in the conventional setting, producing thousands of approximately i.i.d. samples from the conditional distribution often requires millions of forward operator evaluations. This can be prohibitive for complex forward operators, such as those in science and engineering applications based on stochastic differential equations (SDE) or partial differential equations (PDE). Moreover, MCMC schemes are sequential and difficult to parallelize. Beyond the conventional Bayesian setting, one is usually limited to likelihood-free methods; see, e.g., [12]. Among these methods, measure transport provides a general framework to characterize complex posteriors using samples from the joint distribution. The key idea is to construct transport maps that push forward a simple reference (e.g., a standard Gaussian) toward a complex target distribution; see [3] for discussions and reviews. Once obtained, these transport maps provide an immediate way to generate i.i.d. samples from the target distribution by evaluating the map at samples from a reference distribution. Under mild assumptions, there exist infinitely many transport maps that fulfill the push-forward constraint but have drastically different theoretical properties. One way to establish uniqueness is to identify the transport map that satisfies the push-forward constraint and incurs minimal transport cost. Adding transport costs renders the measure transport problem for the conditional distribution into a conditional optimal transport (COT) problem [52]. Solving COT problems is computationally challenging, especially when the number of parameters, n, or the number of measurements, m, are large or infinite. The curse of dimensionality affects methods that use grids or polynomials to approximate transport maps and renders most of them impractical when n + m is larger than ten. Due to their function approximation properties, neural networks are a natural candidate to parameterize COT maps. This choice bridges the measure transport framework and deep generative modeling [38, 27]. While showing promising results, many recent neural network approaches for COT such as [32, 9, 3] rely on adversarial training, which requires solving a challenging stochastic saddle point problem. This paper contributes two neural network approaches for COT that can be trained by maximizing the likelihood of the target samples, and we demonstrate their use in Bayesian inference. Our approaches exploit the known structure of the COT map in different ways. Our first approach parameterizes the map as the gradient of a PICNN [2]; we name it partially convex potential map (PCP-Map). By construction, this yields a monotone map for any choice of network weights. When trained to sufficient accuracy, we obtain the optimal transport map with respect to an expected L2 cost, known as the conditional Brenier map [10]. Our second approach builds upon the relaxed dynamical formulation of the L2 optimal transport problem that instead seeks a map defined by the flow map of an ODE; we name it conditional optimal transport flow (COT-Flow). Here, we parameterize the velocity of the map as the gradient of a scalar potential and obtain a neural ODE. To ensure that the network achieves sufficient accuracy, we monitor and penalize violations of the associated optimality conditions, which are given by a Hamilton Jacobi Bellman (HJB) PDE. A series of numerical experiments are conducted to evaluate PCP-Map and COT-Flow comprehensively. The first experiment demonstrates our approaches’ robustness to hyperparameters and superior numerical accuracy for density estimation compared to results from other approaches in [4] using six UCI tabular datasets [29]. The second experiment demonstrates our approaches’ effectiveness and efficiency by comparing them to a provably convergent approximate Bayesian computation approach on the task of conditional sampling using a Bayesian inference problem involving the stochastic Lotka-Volterra equation, which gives rise to intractable likelihood. The third experiment compares our approaches’ competitiveness against the flow-based neural posterior estimation (NPE) approach studied in [40] on a real-world high-dimensional Bayesian inference problem involving the 1D shallow water equations. The final experiment demonstrates PCP-Map’s improvements in computational stability and efficiency over an amortized version of the approach relevant to [20]. Through these experiments, we conclude that the proposed approaches characterize conditional distributions with improved numerical accuracy and efficiency. Moreover, they improve upon recent computational methods for the numerical solution of the COT problem. Like most neural network approaches, the effectiveness of our methods relies on an adequate choice of network architecture and an accurate solution to a stochastic non-convex optimization problem. As there is little theoretical guidance for choosing the network architecture and optimization hyper-parameters, our numerical experiments show the effectiveness and robustness of a simple random grid search. The choice of the optimization algorithm is a modular component in our approach; we use the common Adam method [24] for simplicity of implementation. The remainder of the paper is organized as follows: Section 2 contains the mathematical formulation of the conditional sampling problem and reviews related learning approaches. Section 3 presents our partially convex potential map (PCP-Map). Section 4 presents our conditional optimal transport flow (COT-Flow). Section 5 describes our effort to achieve reproducible results and procedures for identifying effective hyperparameters for neural network training. Section 6 contains a detailed numerical evaluation of both approaches using six open-source data sets and experiments motivated by Bayesian inference for the stochastic Lotka-Volterra equation and the 1D shallow water equation. Section 7 features a detailed discussion of our results and highlights the advantages and limitations of the presented approaches. This paper is available on arxiv under CC 4.0 license. Authors: (1) Zheyu Oliver Wang, Department of Aeronautics and Astronautics, Massachusetts Institute of Technology, Cambridge, MA and olivrw@mit.edu; (2)  Ricardo Baptista, Computing + Mathematical Sciences, California Institute of Technology, Pasadena, CA and rsb@caltech.edu; (3) Youssef Marzouk, Department of Aeronautics and Astronautics, Massachusetts Institute of Technology, Cambridge, MA and ymarz@mit.edu; (4) Lars Ruthotto, Department of Mathematics, Emory University, Atlanta, GA and lruthotto@emory.edu; (5) Deepanshu Verma, Department of Mathematics, Emory University, Atlanta, GA and deepanshu.verma@emory.edu. This paper is available on arxiv under CC 4.0 license. Authors: Authors: (1) Zheyu Oliver Wang, Department of Aeronautics and Astronautics, Massachusetts Institute of Technology, Cambridge, MA and olivrw@mit.edu; (2)  Ricardo Baptista, Computing + Mathematical Sciences, California Institute of Technology, Pasadena, CA and rsb@caltech.edu; (3) Youssef Marzouk, Department of Aeronautics and Astronautics, Massachusetts Institute of Technology, Cambridge, MA and ymarz@mit.edu; (4) Lars Ruthotto, Department of Mathematics, Emory University, Atlanta, GA and lruthotto@emory.edu; (5) Deepanshu Verma, Department of Mathematics, Emory University, Atlanta, GA and deepanshu.verma@emory.edu. Table of Links Abstract & Introduction Background and Related Work Partially Convex Potential Maps (PCP-Map) for Conditional OT Conditional OT flow (COT-Flow) Implementation and Experimental Setup Numerical Experiments Discussion and ReferenceBayesianInference Abstract & Introduction Abstract & Introduction Background and Related Work Background and Related Work Partially Convex Potential Maps (PCP-Map) for Conditional OT Partially Convex Potential Maps (PCP-Map) for Conditional OT Conditional OT flow (COT-Flow) Conditional OT flow (COT-Flow) Implementation and Experimental Setup Implementation and Experimental Setup Numerical Experiments Numerical Experiments Discussion and Reference BayesianInference Discussion and Reference Abstract. We present two neural network approaches that approximate the solutions of static and dynamic conditional optimal transport (COT) problems, respectively. Both approaches enable sampling and density estimation of conditional probability distributions, which are core tasks in Bayesian inference. Our methods represent the target conditional distributions as transformations of a tractable reference distribution and, therefore, fall into the framework of measure transport. COT maps are a canonical choice within this framework, with desirable properties such as uniqueness and monotonicity. However, the associated COT problems are computationally challenging, even in moderate dimensions. To improve the scalability, our numerical algorithms leverage neural networks to parameterize COT maps. Our methods exploit the structure of the static and dynamic formulations of the COT problem. PCP-Map models conditional transport maps as the gradient of a partially input convex neural network (PICNN) and uses a novel numerical implementation to increase computational efficiency compared to state-of-the-art alternatives. COT-Flow models conditional transports via the flow of a regularized neural ODE; it is slower to train but offers faster sampling. We demonstrate their effectiveness and efficiency by comparing them with state-of-the art approaches using benchmark datasets and Bayesian inverse problems. Key words. Measure transport, generative modeling, optimal transport, Bayesian inference, inverse problems, uncertainty quantification Key words. MSC codes. 62F15, 62M45 MSC codes. Most effective sampling techniques are limited to the conventional Bayesian setting as they require a tractable likelihood model (often given by a forward operator that maps x to y and a noise model) and prior. Even in the conventional setting, producing thousands of approximately i.i.d. samples from the conditional distribution often requires millions of forward operator evaluations. This can be prohibitive for complex forward operators, such as those in science and engineering applications based on stochastic differential equations (SDE) or partial differential equations (PDE). Moreover, MCMC schemes are sequential and difficult to parallelize. Beyond the conventional Bayesian setting, one is usually limited to likelihood-free methods; see, e.g., [12]. Among these methods, measure transport provides a general framework to characterize complex posteriors using samples from the joint distribution. The key idea is to construct transport maps that push forward a simple reference (e.g., a standard Gaussian) toward a complex target distribution; see [3] for discussions and reviews. Once obtained, these transport maps provide an immediate way to generate i.i.d. samples from the target distribution by evaluating the map at samples from a reference distribution. Under mild assumptions, there exist infinitely many transport maps that fulfill the push-forward constraint but have drastically different theoretical properties. One way to establish uniqueness is to identify the transport map that satisfies the push-forward constraint and incurs minimal transport cost. Adding transport costs renders the measure transport problem for the conditional distribution into a conditional optimal transport (COT) problem [52]. Solving COT problems is computationally challenging, especially when the number of parameters, n, or the number of measurements, m, are large or infinite. The curse of dimensionality affects methods that use grids or polynomials to approximate transport maps and renders most of them impractical when n + m is larger than ten. Due to their function approximation properties, neural networks are a natural candidate to parameterize COT maps. This choice bridges the measure transport framework and deep generative modeling [38, 27]. While showing promising results, many recent neural network approaches for COT such as [32, 9, 3] rely on adversarial training, which requires solving a challenging stochastic saddle point problem. This paper contributes two neural network approaches for COT that can be trained by maximizing the likelihood of the target samples, and we demonstrate their use in Bayesian inference. Our approaches exploit the known structure of the COT map in different ways. Our first approach parameterizes the map as the gradient of a PICNN [2]; we name it partially convex potential map (PCP-Map). By construction, this yields a monotone map for any choice of network weights. When trained to sufficient accuracy, we obtain the optimal transport map with respect to an expected L2 cost, known as the conditional Brenier map [10]. Our second approach builds upon the relaxed dynamical formulation of the L2 optimal transport problem that instead seeks a map defined by the flow map of an ODE; we name it conditional optimal transport flow (COT-Flow). Here, we parameterize the velocity of the map as the gradient of a scalar potential and obtain a neural ODE. To ensure that the network achieves sufficient accuracy, we monitor and penalize violations of the associated optimality conditions, which are given by a Hamilton Jacobi Bellman (HJB) PDE. A series of numerical experiments are conducted to evaluate PCP-Map and COT-Flow comprehensively. The first experiment demonstrates our approaches’ robustness to hyperparameters and superior numerical accuracy for density estimation compared to results from other approaches in [4] using six UCI tabular datasets [29]. The second experiment demonstrates our approaches’ effectiveness and efficiency by comparing them to a provably convergent approximate Bayesian computation approach on the task of conditional sampling using a Bayesian inference problem involving the stochastic Lotka-Volterra equation, which gives rise to intractable likelihood. The third experiment compares our approaches’ competitiveness against the flow-based neural posterior estimation (NPE) approach studied in [40] on a real-world high-dimensional Bayesian inference problem involving the 1D shallow water equations. The final experiment demonstrates PCP-Map’s improvements in computational stability and efficiency over an amortized version of the approach relevant to [20]. Through these experiments, we conclude that the proposed approaches characterize conditional distributions with improved numerical accuracy and efficiency. Moreover, they improve upon recent computational methods for the numerical solution of the COT problem. Like most neural network approaches, the effectiveness of our methods relies on an adequate choice of network architecture and an accurate solution to a stochastic non-convex optimization problem. As there is little theoretical guidance for choosing the network architecture and optimization hyper-parameters, our numerical experiments show the effectiveness and robustness of a simple random grid search. The choice of the optimization algorithm is a modular component in our approach; we use the common Adam method [24] for simplicity of implementation. The remainder of the paper is organized as follows: Section 2 contains the mathematical formulation of the conditional sampling problem and reviews related learning approaches. Section 3 presents our partially convex potential map (PCP-Map). Section 4 presents our conditional optimal transport flow (COT-Flow). Section 5 describes our effort to achieve reproducible results and procedures for identifying effective hyperparameters for neural network training. Section 6 contains a detailed numerical evaluation of both approaches using six open-source data sets and experiments motivated by Bayesian inference for the stochastic Lotka-Volterra equation and the 1D shallow water equation. Section 7 features a detailed discussion of our results and highlights the advantages and limitations of the presented approaches.

Efficient Neural Network Approaches for Conditional Optimal Transport: Abstract & Introduction

This paper is under CC 4.0 license. available on arxiv Authors: (1) Joel Jang, CarperAI,University of Washington & Allen Institute for AI; (2) Seungone Kim, KAIST AI; (3) Yizhong Wang, University of Washington; (4) Jack Hessel, University of Washington; (5) Luke Zettlemoyer, Aleph Alpha; (6) Hannaneh Hajishirzi, University of Washington & Allen Institute for AI; (7) Yejin Choi, UC San Diego. 5 CONCLUSION Previous work has shown that adapting LLMs with RLHF helps them generate outputs that are preferred by humans over the supervised fine-tuned counterpart. However, recent work has also pointed out that simply training LLMs to abide by the preference of the general may result in ignoring individual preferences and values. In this work, we provide the first steps to tackle this issue by proposing Reinforcement Learning from Personalized Human Feedback as a multi-objective problem so that LLMs can be aligned to follow conflicting preferences. We propose a promising method called P-SOUPS that is able to composite models trained on single objectives on the fly during inference. We also highlight the scalability of P-SOUPS by showing that it scales linearly, instead of exponentially like the MORL baseline, with regards to the number of new preferences, which is required to provide true personalization to individual users. ACKNOWLEDGMENTS Thanks to Minyoung Hwang, Sungdong Kim, Tim Dettmers, Yoonjoo Lee, and Margaret Li for helpful feedback and discussion. REFERENCES Xiang Ao, Xiting Wang, Ling Luo, Ying Qiao, Qing He, and Xing Xie. PENS: A dataset and generic framework for personalized news headline generation. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), pp. 82–92, Online, August 2021. Association for Computational Linguistics. doi: 10.18653/v1/2021.acl-long.7. URL https: //aclanthology.org/2021.acl-long.7. Yuntao Bai, Andy Jones, Kamal Ndousse, Amanda Askell, Anna Chen, Nova DasSarma, Dawn Drain, Stanislav Fort, Deep Ganguli, T. J. Henighan, Nicholas Joseph, Saurav Kadavath, John Kernion, Tom Conerly, Sheer El-Showk, Nelson Elhage, Zac Hatfield-Dodds, Danny Hernandez, Tristan Hume, Scott Johnston, Shauna Kravec, Liane Lovitt, Neel Nanda, Catherine Olsson, Dario Amodei, Tom B. Brown, Jack Clark, Sam McCandlish, Christopher Olah, Benjamin Mann, and Jared Kaplan. Training a helpful and harmless assistant with reinforcement learning from human feedback. ArXiv, abs/2204.05862, 2022a. URL https://api.semanticscholar.org/ CorpusID:248118878. Yuntao Bai, Saurav Kadavath, Sandipan Kundu, Amanda Askell, Jackson Kernion, Andy Jones, Anna Chen, Anna Goldie, Azalia Mirhoseini, Cameron McKinnon, et al. Constitutional ai: Harmlessness from ai feedback. arXiv preprint arXiv:2212.08073, 2022b. Stephen Casper, Xander Davies, Claudia Shi, Thomas Krendl Gilbert, Jer´ emy Scheurer, Javier ´ Rando, Rachel Freedman, Tomasz Korbak, David Lindner, Pedro Freire, et al. Open problems and fundamental limitations of reinforcement learning from human feedback. arXiv preprint arXiv:2307.15217, 2023. Deborah Cohen, Moonkyung Ryu, Yinlam Chow, Orgad Keller, Ido Greenberg, Avinatan Hassidim, Michael Fink, Yossi Matias, Idan Szpektor, Craig Boutilier, et al. Dynamic planning in openended dialogue using reinforcement learning. arXiv preprint arXiv:2208.02294, 2022. Shachar Don-Yehiya, Elad Venezian, Colin Raffel, Noam Slonim, Yoav Katz, and Leshem Choshen. Cold fusion: Collaborative descent for distributed multitask finetuning. arXiv preprint arXiv:2212.01378, 2022. Yann Dubois, Xuechen Li, Rohan Taori, Tianyi Zhang, Ishaan Gulrajani, Jimmy Ba, Carlos Guestrin, Percy Liang, and Tatsunori B Hashimoto. Alpacafarm: A simulation framework for methods that learn from human feedback. arXiv preprint arXiv:2305.14387, 2023. Shangbin Feng, Chan Young Park, Yuhan Liu, and Yulia Tsvetkov. From pretraining data to language models to downstream tasks: Tracking the trails of political biases leading to unfair nlp models. arXiv preprint arXiv:2305.08283, 2023. Xinyang Geng, Arnav Gudibande, Hao Liu, Eric Wallace, Pieter Abbeel, Sergey Levine, and Dawn Song. Koala: A dialogue model for academic research. Blog post, April, 1, 2023. Conor F Hayes, Roxana Radulescu, Eugenio Bargiacchi, Johan K ˘ allstr ¨ om, Matthew Macfarlane, ¨ Mathieu Reymond, Timothy Verstraeten, Luisa M Zintgraf, Richard Dazeley, Fredrik Heintz, et al. A practical guide to multi-objective reinforcement learning and planning. Autonomous Agents and Multi-Agent Systems, 36(1):26, 2022. Edward J Hu, yelong shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. LoRA: Low-rank adaptation of large language models. In International Conference on Learning Representations, 2022. URL https://openreview.net/forum? id=nZeVKeeFYf9. Chengsong Huang, Qian Liu, Bill Yuchen Lin, Tianyu Pang, Chao Du, and Min Lin. Lorahub: Efficient cross-task generalization via dynamic lora composition. ArXiv, abs/2307.13269, 2023. URL https://api.semanticscholar.org/CorpusID:260155012. Gabriel Ilharco, Marco Tulio Ribeiro, Mitchell Wortsman, Suchin Gururangan, Ludwig Schmidt, Hannaneh Hajishirzi, and Ali Farhadi. Editing models with task arithmetic. ICLR, 2022. Joel Jang, Seungone Kim, Seonghyeon Ye, Doyoung Kim, Lajanugen Logeswaran, Moontae Lee, Kyungjae Lee, and Minjoon Seo. Exploring the benefits of training expert language models over instruction tuning. In Andreas Krause, Emma Brunskill, Kyunghyun Cho, Barbara Engelhardt, Sivan Sabato, and Jonathan Scarlett (eds.), Proceedings of the 40th International Conference on Machine Learning, volume 202 of Proceedings of Machine Learning Research, pp. 14702–14729. PMLR, 23–29 Jul 2023. URL https://proceedings.mlr.press/v202/jang23a. html. Sungdong Kim, Sanghwan Bae, Jamin Shin, Soyoung Kang, Donghyun Kwak, Kang Min Yoo, and Minjoon Seo. Aligning large language models through synthetic feedback. arXiv preprint arXiv:2305.13735, 2023. Hannah Rose Kirk, Bertie Vidgen, Paul Rottger, and Scott A Hale. Personalisation within bounds: A ¨ risk taxonomy and policy framework for the alignment of large language models with personalised feedback. arXiv preprint arXiv:2303.05453, 2023. Noriyuki Kojima, Alane Suhr, and Yoav Artzi. Continual learning for grounded instruction generation by observing human following behavior. Transactions of the Association for Computational Linguistics, 9:1303–1319, 2021. Margaret Li, Suchin Gururangan, Tim Dettmers, Mike Lewis, Tim Althoff, Noah A Smith, and Luke Zettlemoyer. Branch-train-merge: Embarrassingly parallel training of expert language models. arXiv preprint arXiv:2208.03306, 2022. Pan Li and Alexander Tuzhilin. Towards controllable and personalized review generation. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pp. 3237–3245, Hong Kong, China, November 2019. Association for Computational Linguistics. doi: 10.18653/v1/D19-1319. URL https://aclanthology.org/D19-1319. Bodhisattwa Prasad Majumder, Shuyang Li, Jianmo Ni, and Julian McAuley. Generating personalized recipes from historical user preferences. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pp. 5976–5982, Hong Kong, China, November 2019. Association for Computational Linguistics. doi: 10.18653/v1/D19-1613. URL https://aclanthology.org/D19-1613. Pierre-Emmanuel Mazare, Samuel Humeau, Martin Raison, and Antoine Bordes. Training mil- ´ lions of personalized dialogue agents. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pp. 2775–2779, Brussels, Belgium, OctoberNovember 2018. Association for Computational Linguistics. doi: 10.18653/v1/D18-1298. URL https://aclanthology.org/D18-1298. Jacob Menick, Maja Trebacz, Vladimir Mikulik, John Aslanides, Francis Song, Martin Chadwick, Mia Glaese, Susannah Young, Lucy Campbell-Gillingham, Geoffrey Irving, et al. Teaching language models to support answers with verified quotes. arXiv preprint arXiv:2203.11147, 2022. Reiichiro Nakano, Jacob Hilton, S. Arun Balaji, Jeff Wu, Ouyang Long, Christina Kim, Christopher Hesse, Shantanu Jain, Vineet Kosaraju, William Saunders, Xu Jiang, Karl Cobbe, Tyna Eloundou, Gretchen Krueger, Kevin Button, Matthew Knight, Benjamin Chess, and John Schulman. Webgpt: Browser-assisted question-answering with human feedback. ArXiv, abs/2112.09332, 2021a. URL https://api.semanticscholar.org/CorpusID:245329531. Reiichiro Nakano, Jacob Hilton, Suchir Balaji, Jeff Wu, Long Ouyang, Christina Kim, Christopher Hesse, Shantanu Jain, Vineet Kosaraju, William Saunders, et al. Webgpt: Browser-assisted question-answering with human feedback. arXiv preprint arXiv:2112.09332, 2021b. Long Ouyang, Jeff Wu, Xu Jiang, Diogo Almeida, Carroll L. Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, John Schulman, Jacob Hilton, Fraser Kelton, Luke E. Miller, Maddie Simens, Amanda Askell, Peter Welinder, Paul Francis Christiano, Jan Leike, and Ryan J. Lowe. Training language models to follow instructions with human feedback. ArXiv, abs/2203.02155, 2022a. URL https://api.semanticscholar.org/ CorpusID:246426909. Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, et al. Training language models to follow instructions with human feedback. Advances in Neural Information Processing Systems, 35: 27730–27744, 2022b. Baolin Peng, Chunyuan Li, Pengcheng He, Michel Galley, and Jianfeng Gao. Instruction tuning with gpt-4. arXiv preprint arXiv:2304.03277, 2023. Jonas Pfeiffer, Naman Goyal, Xi Victoria Lin, Xian Li, James Cross, Sebastian Riedel, and Mikel Artetxe. Lifting the curse of multilinguality by pre-training modular transformers. arXiv preprint arXiv:2205.06266, 2022. Alexandre Rame, Guillaume Couairon, Mustafa Shukor, Corentin Dancette, Jean-Baptiste Gaya, Laure Soulier, and Matthieu Cord. Rewarded soups: towards pareto-optimal alignment by interpolating weights fine-tuned on diverse rewards. arXiv preprint arXiv:2306.04488, 2023. Alireza Salemi, Sheshera Mysore, Michael Bendersky, and Hamed Zamani. Lamp: When large language models meet personalization. arXiv preprint arXiv:2304.11406, 2023. Shibani Santurkar, Esin Durmus, Faisal Ladhak, Cinoo Lee, Percy Liang, and Tatsunori Hashimoto. Whose opinions do language models reflect? arXiv preprint arXiv:2303.17548, 2023. John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, and Oleg Klimov. Proximal policy optimization algorithms. arXiv preprint arXiv:1707.06347, 2017. Umer Siddique, Paul Weng, and Matthieu Zimmer. Learning fair policies in multi-objective (deep) reinforcement learning with average and discounted rewards. In International Conference on Machine Learning, pp. 8905–8915. PMLR, 2020. David Silver, Satinder Singh, Doina Precup, and Richard S Sutton. Reward is enough. Artificial Intelligence, 299:103535, 2021. Prasann Singhal, Tanya Goyal, Jiacheng Xu, and Greg Durrett. A long way to go: Investigating length correlations in rlhf. arXiv preprint arXiv:2310.03716, 2023. Feifan Song, Bowen Yu, Minghao Li, Haiyang Yu, Fei Huang, Yongbin Li, and Houfeng Wang. Preference ranking optimization for human alignment. arXiv preprint arXiv:2306.17492, 2023. Nisan Stiennon, Long Ouyang, Jeffrey Wu, Daniel Ziegler, Ryan Lowe, Chelsea Voss, Alec Radford, Dario Amodei, and Paul F Christiano. Learning to summarize with human feedback. Advances in Neural Information Processing Systems, 33:3008–3021, 2020. Alane Suhr and Yoav Artzi. Continual learning for instruction following from realtime feedback. arXiv preprint arXiv:2212.09710, 2022. Richard S Sutton and Andrew G Barto. Reinforcement learning: An introduction. MIT press, 2018. Romal Thoppilan, Daniel De Freitas, Jamie Hall, Noam Shazeer, Apoorv Kulshreshtha, Heng-Tze Cheng, Alicia Jin, Taylor Bos, Leslie Baker, Yu Du, et al. Lamda: Language models for dialog applications. arXiv preprint arXiv:2201.08239, 2022. Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timothee´ Lacroix, Baptiste Roziere, Naman Goyal, Eric Hambro, Faisal Azhar, et al. Llama: Open and ` efficient foundation language models. arXiv preprint arXiv:2302.13971, 2023. Peter Vamplew, Richard Dazeley, Cameron Foale, Sally Firmin, and Jane Mummery. Humanaligned artificial intelligence is a multiobjective problem. Ethics and Information Technology, 20:27–40, 2018. Kristof Van Moffaert and Ann Nowe. Multi-objective reinforcement learning using sets of pareto ´ dominating policies. The Journal of Machine Learning Research, 15(1):3483–3512, 2014. Kristof Van Moffaert, Madalina M Drugan, and Ann Nowe. Scalarized multi-objective reinforce- ´ ment learning: Novel design techniques. In 2013 IEEE symposium on adaptive dynamic programming and reinforcement learning (ADPRL), pp. 191–199. IEEE, 2013. Yizhong Wang, Hamish Ivison, Pradeep Dasigi, Jack Hessel, Tushar Khot, Khyathi Raghavi Chandu, David Wadden, Kelsey MacMillan, Noah A Smith, Iz Beltagy, et al. How far can camels go? exploring the state of instruction tuning on open resources. arXiv preprint arXiv:2306.04751, 2023. Mitchell Wortsman, Suchin Gururangan, Shen Li, Ali Farhadi, Ludwig Schmidt, Michael Rabbat, and Ari S Morcos. lo-fi: distributed fine-tuning without communication. arXiv preprint arXiv:2210.11948, 2022a. Mitchell Wortsman, Gabriel Ilharco, Samir Ya Gadre, Rebecca Roelofs, Raphael Gontijo-Lopes, Ari S Morcos, Hongseok Namkoong, Ali Farhadi, Yair Carmon, Simon Kornblith, et al. Model soups: averaging weights of multiple fine-tuned models improves accuracy without increasing inference time. In International Conference on Machine Learning, pp. 23965–23998. PMLR, 2022b. Jeff Wu, Long Ouyang, Daniel M Ziegler, Nisan Stiennon, Ryan Lowe, Jan Leike, and Paul Christiano. Recursively summarizing books with human feedback. arXiv preprint arXiv:2109.10862, 2021a. Yuwei Wu, Xuezhe Ma, and Diyi Yang. Personalized response generation via generative split memory network. In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pp. 1956–1970, 2021b. Zeqiu Wu, Yushi Hu, Weijia Shi, Nouha Dziri, Alane Suhr, Prithviraj Ammanabrolu, Noah A Smith, Mari Ostendorf, and Hannaneh Hajishirzi. Fine-grained human feedback gives better rewards for language model training. arXiv preprint arXiv:2306.01693, 2023. Jie Xu, Yunsheng Tian, Pingchuan Ma, Daniela Rus, Shinjiro Sueda, and Wojciech Matusik. Prediction-guided multi-objective reinforcement learning for continuous robot control. In International conference on machine learning, pp. 10607–10616. PMLR, 2020. Jing Xu, Arthur Szlam, and Jason Weston. Beyond goldfish memory: Long-term open-domain conversation. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 5180–5197, Dublin, Ireland, May 2022. Association for Computational Linguistics. doi: 10.18653/v1/2022.acl-long.356. URL https: //aclanthology.org/2022.acl-long.356. Runzhe Yang, Xingyuan Sun, and Karthik Narasimhan. A generalized algorithm for multi-objective reinforcement learning and policy adaptation. Advances in neural information processing systems, 32, 2019. Saizheng Zhang, Emily Dinan, Jack Urbanek, Arthur Szlam, Douwe Kiela, and Jason Weston. Personalizing dialogue agents: I have a dog, do you have pets too? arXiv preprint arXiv:1801.07243, 2018. Lianmin Zheng, Wei-Lin Chiang, Ying Sheng, Siyuan Zhuang, Zhanghao Wu, Yonghao Zhuang, Zi Lin, Zhuohan Li, Dacheng Li, Eric Xing, et al. Judging llm-as-a-judge with mt-bench and chatbot arena. arXiv preprint arXiv:2306.05685, 2023. Yinhe Zheng, Guanyi Chen, Minlie Huang, Song Liu, and Xuan Zhu. Personalized dialogue generation with diversified traits. arXiv preprint arXiv:1901.09672, 2019. Daniel M Ziegler, Nisan Stiennon, Jeffrey Wu, Tom B Brown, Alec Radford, Dario Amodei, Paul Christiano, and Geoffrey Irving. Fine-tuning language models from human preferences. arXiv preprint arXiv:1909.08593, 2019. A DETAILS OF EVALUATION SETUP We use a modified version of the GPT4 annotation prompt used by Dubois et al. (2023). We modify the criteria to perform the pairwise evaluation from general to a single preference dimension. We also provide 3 demonstrations: one scenario where there is a tie because both responses do not contain any notion of the preference (e.g. both responses do not show any signs of friendliness), one scenario where there is a clear winner, and one scenario where they are both good, but one is better than the other. Simulated pairwise evaluation We recruited 24 crowd workers for our human evaluation. Figure 5 shows the interface used for human evaluation. We consider both the ‘Tie’ and ‘Both are bad’ options to be Ties. B CRITERIA-WISE EVALUATION The criteria-wise win rate (%) across all the methods are shown in Figure 6. The criteria-wise win rate was calculated by getting the average win-rate of the preference combinations that contained the specific preference dimension. For example, when calculating the criteria-wise win rate of ‘Elementary’, we got the average win-rate of the preference combinations that contained the ‘Elementary’ preference, which includes AAA, AAB, ABA, and ABB. C THE FULL LIST OF EVALUATION PROMPTS The full list of evaluation prompts used in our experiments are provided in Table 5 D DETAILED RESULTS FOR HUMAN EVALUATION AND GPT-4 EVALUATION We provide detailed results (win / loss / tie for each of the preference combinations for our main experimental results. Table 6 shows the GPT4 evaluation and Table 7 shows the human evaluation results. E EXAMPLES OF P-SOUPS TEXT GENERATIONS Table 8 shows empirical examples of the text generated from each preference combination of the 16 preference combination experiments for the same prompt.

Personalized Soups: LLM Alignment Via Parameter Merging - Conclusion & References

This paper is under CC 4.0 license. available on arxiv Authors: (1) Joel Jang, CarperAI,University of Washington & Allen Institute for AI; (2) Seungone Kim, KAIST AI; (3) Yizhong Wang, University of Washington; (4) Jack Hessel, University of Washington; (5) Luke Zettlemoyer, Aleph Alpha; (6) Hannaneh Hajishirzi, University of Washington & Allen Institute for AI; (7) Yejin Choi, UC San Diego. 2 RELATED WORK Incorporating human preference feedback into a reward model, and subsequently optimizing a language model to output text that reward model scores highly with an RL algorithm, has been shown to result in language models that generate outputs humans generally prefer (Ouyang et al., 2022b). This process has been applied to summarization (Ziegler et al., 2019; Stiennon et al., 2020; Wu et al., 2021a), answering questions with long-form answers using text retrieved from the web (Nakano et al., 2021b; Menick et al., 2022), generating engaging responses in a dialogue settings (Thoppilan et al., 2022; Cohen et al., 2022) and following human instructions (Kojima et al., 2021; Suhr & Artzi, 2022; Kim et al., 2023). Aligning Language Models To Human Preferences However, the standard RLHF setup commonly addressed in prior work assumes a reward model that accounts only for average annotator preference, i.e., the fact that different users may desire different outputs, even for the same prompt, is ignored Casper et al. (2023). Individual preferences can vary not only on aesthetic axes, but also on semantics. For example, Santurkar et al. (2023) use public opinion polling to show that “default” LLM preferences vary in their degree of expressed-opinion alignment with different average opinions among demographic groups.[2] Kirk et al. (2023) defines a taxonomy and policy framework for the alignment of LLMs with personalized feedback. While Wu et al. (2023) performs fine-grained RLHF which is very similar in spirit and allows personalization, our work develops MORL algorithms for scenarios where there are conflicting preferences, not only orthogonal objectives. In this work, we propose formulating LLM personalization as a MORL problem, which was typically studied in decision-making tasks (Hayes et al., 2022) that aims to tackle the problem of simply optimizing by a single, scalar, additive reward function (Sutton & Barto, 2018), which possesses many limitations such as (1) suboptimal solutions due to lack of representation (Hayes et al., 2022), (2) lack of explainability of distinct objectives, and (3) ensuring fair outcomes for multiple participants (Vamplew et al., 2018; Siddique et al., 2020). Multi-objective Reinforcement Learning (MORL) Previous work has aimed to alleviate these problems through novel MORL methods (Van Moffaert et al., 2013; Van Moffaert & Nowe´, 2014; Yang et al., 2019; Xu et al., 2020). Other work aims to solve complex problems such as water management, military purchasing, wind farm control, etc. (Hayes et al., 2022) by converting the single-objective RL problem into a MORL problem. In this work, we convert the problem of aligning LLMs to human preferences into a MORL problem to (1) provide a more optimal solution for each individual, (2) allow users to dynamically choose the distinct objectives they want to optimize, and (3) ensure fairness by allowing preferences that may be in the long-tail to be integrated. Personalization in Natural Language Processing (NLP) has mainly been focused on creating personalized dialogue agents (Zhang et al., 2018; Mazare et al. ´ , 2018; Zheng et al., 2019; Wu et al., 2021b; Xu et al., 2022), where the task is to create chitchat agents that are engaging with distinct personas based on user profile (e.g. gender, age, residence, etc.) or past user history data (e.g. Reddit posts, etc.). Another line of work (Salemi et al., 2023) leverages personalized information to boost performance on specific tasks such as review generation (Li & Tuzhilin, 2019), recipe generation (Majumder et al., 2019), and headline generation (Ao et al., 2021). This line of work requires model providers to make better models utilizing the personal information of the user. In our work, we propose a framework that allows users to choose which preference the language model should prefer, essentially giving control to the user. Personalization in Natural Language Processing Recent work has shown that performing weighted linear interpolation of model parameters leads to the composition of each model ability (Li et al., 2022; Wortsman et al., 2022b;a; Don-Yehiya et al., 2022; Huang et al., 2023). This line of work has led to many interesting applications of model merging such as composing the abilities of expert models that perform different tasks (Ilharco et al., 2022; Jang et al., 2023) and introducing language-specific modules for growing the total capacity of multilingual LMs (Pfeiffer et al., 2022). Parameter Merging Most recently, Rame et al. (2023) proposed to merge policy models that were trained to perform specific tasks such as question answering and summarization using proxy reward models. While they mostly deal with reward models trained on the same data, our proposed MORL methods are an extension of this work that actually deals with diverse reward models trained on multifaceted human feedback to show compositional abilities through parameter merging rather than just ensembling. [2] Feng et al. (2023) suggests that “default” LLM expressed opinions stem directly from the pretraining data.

Personalized Soups: LLM Alignment Via Parameter Merging - Related Work

Rosetta Stone and machine-interpretability: Wikidata and a modeling paradigm 

This paper is available on arxiv under CC 4.0 license. Authors: (1) Vogt, Lars, TIB Leibniz Information Centre for Science and Technology; (2) Konrad, Marcel, TIB Leibniz Information Centre for Science and Technology; (3) Prinz, Manuel, TIB Leibniz Information Centre for Science and Technology. Table of Links Abstract & Introduction Interoperability Semantic interoperability and what natural languages like English can teach us Requirements for successfully communicating terms and statements Parallels between the structure of natural language statements and data schemata with implications for semantic interoperability What makes a term a good term and a schema a good schema? The need for a machine-actionable Rosetta Stone for (meta)data that acts as an interlingua for specifying reference terms and reference schemata to support cognitive and semantic interoperability Rosetta Stone and machine-readability: UPRIs, XML Schema datatypes, and RDF for communicating terms, datatypes, and statements Rosetta Stone and machine-interpretability: Wikidata and a modeling paradigm for (meta)data statements based on English Rosetta Stone and semantic interoperability: Specifying term mappings and schema crosswalks Rosetta Stone and cognitive interoperability: Specifying display templates and using a query builder Discussion Related work Conclusion, Acknowledgements, & References The need for a machine-actionable Rosetta Stone for (meta)data that acts as an interlingua for specifying reference terms and reference schemata to support cognitive and semantic interoperability Above, we discussed the role of terms and statement structures (i.e., syntax trees and (meta)data schemata) in reliably communicating the meaning and thus the semantic content of (meta)data statements. Statement structures specify syntactic positions or slots with semantic roles or constraint specifications for a given statement type. To achieve semantic interoperability, we therefore need controlled vocabularies (i.e., ) and across ontologies for FAIR terms and their . And we need a schemata and and ontologies ontological and referential term mappings terminological interoperability (meta)dat ontological and referential schema crosswalks for FAIR (meta)data statements their schematic interoperability. We also discussed why we think that it is impossible to agree on a best term for every possible type of entity and a best schema for every possible type of statement, due to varying frames of reference and operational priorities. Therefore, we think that we need something like a to support the establishment of semantic interoperability across different terms and different schemata for a given type of (meta)data statement. This Rosetta Stone needs to function like an , with which term mappings and schema crosswalks can be easily specified and operationalized. The building blocks of the are and Each entity type must have specified a corresponding reference term, and each statement type must have a corresponding reference schema. Terms from controlled vocabularies can be mapped to their corresponding reference term, and schemata to their corresponding reference schema. Constraint specifications for slots of reference schemata must refer to reference terms in the case of resources, and to reference datatype specifications in the case of values. These three types of building blocks take over the role of , so that it would no longer be necessary to specify schema crosswalks for every possible pair of schemata of a given type of (meta)data statement and to specify term mappings for every possible pair of terms. This would minimize the number of schema crosswalks and term mappings that need to be specified in order to achieve schematic and terminological interoperability for a given type of statement (Fig. 4). machine-actionable Rosetta Stone interlingua interlingua reference terms, reference datatype specifications, reference schemata. mediating connectors Ideally, a reference schema is based on a generic Rosetta modeling paradigm that allows the reconstruction of the natural language statement underlying the datum. At the same time, it should document this statement using a formalized structure to ensure its human- and machine-actionability. With respect to human-actionability, the Rosetta modeling paradigm should reflect as closely as possible the structure of natural language statements, favoring lean over complex models, with the aim of reducing overall modeling complexity and modeling burden. Many schemata are very complex and include positions with resources that do not directly align with any input slot (e.g., and in Fig. 2E). Such schemata are not suitable for use as reference schemata. ‘scalar measurement datum’ ‘scalar value specification’ Schemata that conform to the Rosetta modeling paradigm should be easy to understand and to apply, allowing any producer of (meta)data to specify new reference schemata for types of statements that do not yet have a reference schema assigned to them, and allowing any application developer to readily use their (meta)data. It should not require experience in semantics and knowledge engineering on the part of the data producer and the application developer. With regard to the structure of reference schemata, we need to consider the machine-actionability of the resulting (meta)data statements. In other words, we need to consider which operations are important for such a reference schema. While reasoning is important for domain knowledge, especially when developing ontologies, other types of operations such as searching and exploring are more important in the context of empirical (meta)data and (meta)data management in general. However, regardless of the choice of operations and associated tools, the application of reference schemata must result in FAIR (meta)data that are machine-readable and machine-interpretable in order to be machine-actionable. As for reference terms, they should ideally be collected in a large, controlled cross-domain vocabulary and should be machine-readable and machine-interpretable to be machine-actionable.

SCALAR

scalar-labs.com

1-10 employees

Since 2017

COMPANY RANKING

SCALAR

EVERGREEN INDEX #9966

Representing this company? Click here to claim and customize this page!

Scalar WIKI

Apply to get your Tech Company News Page live on HackerNoon.com today!

SCALAR

scalar-labs.com

1-10 employees

Since 2017

COMPANY RANKING

SCALAR

EVERGREEN INDEX #9966

RELATED COMPANIESRANK

STORIES ABOUT SCALAR

Representing this company? Click here to claim and customize this page!

Scalar WIKI

Apply to get your Tech Company News Page live on HackerNoon.com today!