Best match

Most recent

Einstein's theory of gravity—general relativity—has been very successful for more than a century. However, it has theoretical shortcomings. This is not surprising: the theory predicts its own failure at spacetime singularities inside black holes—and the Big Bang itself.

https://www.bing.com/th?id=OVFT.vI0lU_sBqOoYvpDiaXg3_y&pid=News

Why Einstein must be wrong: In search of the theory of gravity

https://www.bing.com/th?id=ODF.z8AwA_Nu7cBuQBwIIezAVw&pid=news

Phys.org

Destiny

Since the launch of Destiny 2's Season of the Witch, one craftible weapon has steadily climbed to top of the game's competitive and casual PvP meta.

https://www.bing.com/th?id=OVFT.LJQ5sgwkUAeQv8DGrH5dwC&pid=News

Nintendo

How One Destiny 2 Gun Rose to the Top of the Meta

https://www.bing.com/th?id=ODF.NvtItL5e7oo1bQn4ODytnw&pid=news

Game Rant

In addition to this list, there are a number of constructs that can be classified as an expression but do not follow any general syntax rules. These generally have the semantics of a function or operator and are explained in the appropriate location in ...

4.2. Value Expressions

PostgreSQL

Einstein's theory of relativity say black holes are 'bald', but a new tweak to his research may give the mysterious objects their long-sought 'hair.'

https://www.bing.com/th?id=OVFT.mSRLEnulytaVn-gejp3SOS&pid=News

'Twisty' new theory of gravity says information can escape black holes after all

https://www.bing.com/th?id=ODF.nYADEgS75l8rdCg9D-p_OQ&pid=news

YAHOO!News

[Steven Merrifield] built his own Scalar Network Analyzer and it’s a beauty! [Steve]’s SNA has a digital pinout matching a Raspberry Pi, but any GPIO could be used to operate the device and ...

Scalar Network Analyzer

Hackaday

The end result: The scalar fields added to general relativity, when explored through the teleparallel lens, gave black holes some hair. The "hair" in this case is the presence of a strong scalar ...

Live Science

Get the most recent info and news about Scalar on HackerNoon, where 10k+ technologists publish stories for 4M+ monthly readers.

The World's Reliable Database - 高い信頼性を有する独自のデータベースを活用してデジタライゼーションの加速を支援します

Scalar

Let be a cyclic group, is the element in the group, is the scalar, and the Multi-Scalar Multiplication (MSM) is the algorithm for calculating , the sum of multiple scalar multiplications. As the group operation is more complicated than the addition and multiplication of elements in the finite field, the MSM algorithm is employed to reduce the number of times of group operation as much as possible. Usually, is a cyclic group defined on the elliptic curve of on the finite field of , with the order being bits, and . If the group operation is conducted by the plain algorithm of fast exponentiation, the number of group operations needed for every is times on average, and for all is times in total. Next, we will introduce some optimization methods corresponding to larger . G P_i ∈ G a_i ∈ Z ∑_{i=1}^{n} a_{i} P_{i} G y^2=x^3+ax+b F_p | | G b ≈ 256 0 ≤ a_i< 2^b a_i.P_i 1.5bn a_i.P_i 1.5b n Optimization Based on the Windowing Technique We can split the bit scalar into c width windows: reduce the scalar to base system. b 2^c Here, } , so we reduce the original algorithm issue of -bit MSM into a smaller -bit issue. We can put the same scalars from into a bucket and write them in another form: Q_j=∑_{i=1}^{n a_{i j} P_{i} b c ∑_{i=1}^{n} a_{i j} P_{i} For example, when , it is calculated as: c = 3 When calculating , set the partial sum T_l=S_l+⋯+S_k, then ∑kS_k However, the operation results of can be obtained through the recurrence sequence . It means that only one group operation is needed for the operation for each . Under the MSM algorithm with cc-bit scalar, all requires times of group operation. All requires times and the sum of these requires times of group operation. Therefore, each window with the value of c requires times of group operations. The operation of } only requires the normal method of fast exponentiation by times of group operation. T_l T_l=T_l+1+S_l T_l S_k n−2^c+1 T_k 2^ 1 c− T_k ^c−2 2 n+2^c−2 ∑_{j=0}^{b / c-1} Q_{j} 2^{j c (b/c−1)(c+1)=b − c + b/c−1 In summary, we can conclude that the total number of group operations required by the method of the Windowing Technique of a width of is c Set , then the number of group operations is . When , the number of group operations reduces to of its original value. c = logn 2bn/logn n≈10^5 1/12 Optimization Based on Group Endomorphism For the cyclic group on the elliptic curve on the finite field F_p, if the following group endomorphism of φ can be found: there exists , satisfying holds for all the points on , then it is easy to prove that such an endomorphism is a multiplication map, namely there is a making hold for all the points of on , which means that whenever we know the coordinates of one point, we can change it to the coordinates of another point simply by multiplying a number in to both the values of abscissa and ordinate. The algorithm can be further optimized based on this important property. G y^2=x^3+ax+b α,β ∈ F_p φ(x,y)=(αx,βy) G λ φ(P)=λP P G F_p When . The reverse value of one point can be obtained only by taking the opposite number from the ordinate. Based on this feature, in the plain fast exponentiation algorithm, the scalar can be written into the non-adjacent form (NAF), namely and any two adjacent cannot be non-zero at the same time. Compared to the -bit scalars, the group operation number in the fast exponentiation algorithm can be reduced from the original average of times to times. This technique can also be used to optimize the Windowing Technique. After writing to λ = −1, α = 1, β = −1 e_{i}.2^{i}, e_{i} ∈ {−1,0,1} ∑ e_i b 3/2⋅b 4/3⋅b a_i a_i=∑_{j=0}^{b / c-1} a_{i, j} 2^{j c}, 0 <= a_{i, j}<2^{c} conduct the following operation: The result of and - can be obtained. Since the complexity of addition and subtraction in group operation of an elliptic curve is the same, we can put these terms into a bucket according to the absolute value of scalars. Therefore, the number of groups can be reduced from the original 2c to , and the overall number of group operations required is reduced from (1) to: a_i=∑_{j=0}^{b / c-1} a’_{i, j}^2^{j c} 2^{c-1} <= a’_{i, j}^ <= 2^{c-1} 2^c If the parameters of the elliptic curve are special, for example, the BLS curve can be written as , and , take third-order element α from F_p^x, then there is a corresponding that holds y^2=x^3+b p ≡ 1( mod3 ) λ λ( )=( ) x,y αx,y Algorithm 1: Signed Digits In addition, . Then the multiplication operation can be optimized as follows: λ3 ≡ 1(mod |G| ) From [1], we can find a small enough scalar | |, | |<√ to make the above equation true, which gives us a way to reduce the multiplication of bb bits into the multiplication of two bits. Use it into the Windowing Technique, and the number of group operations is reduced to m1 m2 G b/2 In summary, both the above two optimization methods can reduce the number of group operations by when . 5.5% n=10^5 References [1] Francesco Sica, Mathieu Ciet, and Jean-Jacques Quisquater. Analysis of the gallant-lambert-vanstone method based on eﬀicient endomorphisms: Elliptic and hyperelliptic curves. In International Workshop on Selected Areas in Cryptography, pages 21–36. Springer, 2002.

Optimization of Multi-Scalar Multiplication Algorithm: Sin7Y Tech Review (21)

Today is the first day of the WunderGraph launch week, it's our first launch week actually. We have a lot of things to share with you, like Open Federation, Cosmo, and more... What excites me the most is that today marks the beginning of a new era for . For a very long time, the GraphQL ecosystem has been dominated by a single company. However, this is about to change through two initiatives and the ecosystem that will emerge around them. GraphQL The first step is , an MIT-Licensed specification to build federated GraphQL APIs. The goal of Open Federation is to enable the GraphQL ecosystem to build federated GraphQL APIs without having to rely on Apollo's proprietary technology. It's a drop-in replacement for Apollo Federation, it's open source and free to use for everyone. Open Federation Most importantly, we're not announcing Open Federation alone. Open Federation is a joint effort by WunderGraph, Grafbase, Escape Tech, Neo4j, TailorTech, Sibipro, Soundtrack Your Brand, Travelpasses, and many more. The second step is , a specification for composed GraphQL APIs that goes way beyond what Apollo Federation offers, like support for composing not just GraphQL APIs, but also REST(OpenAPI), gRPC, AsyncAPI, and more. GraphQL Fusion . GraphQL Fusion will live under the GraphQL Foundation The goal of Fusion is to enable the composition of GraphQL APIs with a diverse set of vendors and technologies. Fusion is a joint effort by ChilliCream, The Guild, Hasura, IBM, Solo.io, AWS AppSync, and WunderGraph. Instead of every vendor building their own proprietary composition technology, we're building a single open-source specification that everyone can implement, so that we can all work together to build a better GraphQL ecosystem. For today, we're focusing on Open Federation, but stay tuned for more information about GraphQL Fusion. You'll definitely hear more about it during the GraphQL Conference by the GraphQL Foundation. We'll be there as well, so make sure to say hi at our booth. The History of Open Federation I'm the creator of graphql-go-tools, a GraphQL library for Go that is used by many companies to build GraphQL Gateways and other GraphQL tools. I've been working on this library for more than 5 years now and it has been a great success. Almost 3 years ago, I started adding support for . As excited as I was about the idea of Federation, the community was not ready for it yet. I added support for Subscriptions years ago, but demand for it was very low, so my focus shifted to solving other problems. Apollo Federation to graphql-go-tools Fast forward to 2023, and suddenly things have changed fundamentally. The GraphQL ecosystem has grown a lot and the demand for Federation has increased significantly. At the same time, Apollo started closing down their open-source ecosystem with license changes, price increases, and more. All of this led to a lot of people reaching out to us and asking for help. Some of them, like TailorTech, were even willing to sponsor the development of a Federation v2 compatible Gateway and composition tooling for graphql-go-tools. Big enterprises helped us a lot with testing or providing GraphQL schemas of their federated Graphs to test our implementation. But there was a problem we had to overcome first. What actually is Apollo Federation? This might sound like a silly question, but it's actually not. At this point, we should all have some high-level understanding of what the Federation is. But have you ever looked at the details, like the specifications? Here's the of Apollo Federation v2. It's nine Directives and one Scalar, that's it. No comments, no explanations, no examples, nothing. It's everything but a specification and it actually leaves all the important questions unanswered. "Specification" For example, how do you actually implement a Federation Gateway? What is the Gateway supposed to do? What are the rules to validate a federated GraphQL schema? How do you actually compose federated GraphQL APIs? What are the rules to check if two or more subgraphs are compatible? In comparison, the GraphQL Specification describes in a lot of detail how a GraphQL Server should behave. It contains numerous examples for validation and execution rules, as well as descriptions of basic algorithms like how to merge selection sets or how to execute a GraphQL Query. If you want to understand Apollo Federation in detail, you have to read a lot of documentation in different places and even then, you'll still have a lot of questions. You might be able to reverse-engineer the behavior of the Apollo Gateway and Router, but even if you do, there are still some behaviors that seem random and are not documented anywhere. So we asked ourselves, how can we build tools to compose subgraphs and a Router / Gateway to resolve federated GraphQL Queries without having a proper description of "our understanding" of Federation? The birth of Open Federation That's when we decided to start writing down our own understanding of the Federation. By writing down all our knowledge, you're able to criticize it and help us improve it. If you find gaps or mistakes, we can collaborate to find a common understanding of what Federation is, how composition works, and how a Gateway should behave. From that point on, multiple vendors can implement the same specification so you're free to choose the best solution for your use case. You're not locked into a single vendor anymore, you can switch to a different vendor at any time. You can even if you want to. In addition, you can suggest improvements to the specification and actually have an impact on the future of federated GraphQL APIs. build your own Gateway The goals of Open Federation We don't believe that there's a future for proprietary API Composition solutions. It's very unlikely that a single vendor with a closed-source solution is capable of serving the market with the best possible solution. We've seen the opposite in the past with OpenAPI. Before OpenAPI, there were many proprietary custom solutions to describe and manage . However, OpenAPI has become the de-facto standard for REST APIs and is supported by many vendors and tools. It's a great success story and has led to a lot of innovation in the REST API space. REST APIs We believe that Open Federation can have a similar impact on the GraphQL ecosystem. An open specification will make federated GraphQL much more accessible and attractive to a lot of companies and therefore will lead to a bigger adoption of GraphQL in general. This in turn will attract more vendors to the GraphQL ecosystem, which will lead to more innovation and healthy competition. After all, it's in the best interest of the GraphQL ecosystem to have a diverse set of vendors and technologies. By standardizing the composition of GraphQL APIs, vendors can focus on their strengths and collaborate with complementing solutions. The key is Composable Architecture and interoperability. Different implementations of the specification can focus on different use cases and technologies. One vendor might focus on a Gateway that is optimized for performance by using Rust or Go, while another vendor might optimize for extensibility, e.g. by writing the Gateway in NodeJS. At this point, you might be wondering, what's the difference between Open Federation and Apollo Federation? What's the difference between Open Federation and Apollo Federation? Apollo Federation is not a complete specification. It's a set of directives that, if used correctly on a subgraph, will make it compatible with the Apollo Gateway and Router. Open Federation builds on top of the same directives to specify subgraphs. It's a specification that describes in detail the rules to compose subgraphs, and how federated GraphQL requests should be planned and executed. Open Federation can be seen as a drop-in replacement for Apollo Federation, just that it's way more specific, open source, and free to use for everyone. What's the relationship between Open Federation and GraphQL Fusion? For a lot of use cases, federated GraphQL APIs will be more than enough, and there's no reason not to adopt or use Open Federation. However, there will be use cases where you want to compose not just federated GraphQL APIs, but also REST APIs, gRPC APIs, AsyncAPI, and more. This is where GraphQL Fusion comes into play. Conclusion Federated GraphQL is a mature technology and has been used in production for years. GraphQL Fusion is still in its early stages and will take some time to get to the same level of maturity. We bet on both technologies and believe that they will complement each other very well. Time will tell if both technologies will live alongside each other or if one will eventually dominate the other. What matters most to us is that we're standardizing the composition of APIs to build a better GraphQL ecosystem. Also published . here If you haven't already, make sure to check out and have a look at the specifications. We're looking forward to your feedback and contributions. Open Federation

Open Federation - An MIT-Licensed Specification to Build Federated GraphQL APIs

Logging in Azure with Application Insights and Serilog

I recently went through a very detailed and well-explained Python-based project/lesson by which is called . This is a tiny scalar-valued autograd engine with a neural net on top of it. This video explains how to build such a network from scratch. karpathy micrograd https://www.youtube.com/watch?v=VMj-3S1tku0&embedable=true The project above is, as expected, built on . For learning purposes, I wanted to see how such a network may be implemented in and came up with a 🤖 repository (and also with a of how the network may be trained). Python TypeScript micrograd-ts demo Trying to build anything on your own very often gives you a much better understanding of a topic. So, this was a good exercise, especially taking into account that the whole code is just  ~200 lines of TS code with no external dependencies. The micrograd-ts repository might be useful for those who want to get a basic understanding of how work, using a TypeScript environment for experimentation. neural networks With that being said, let me give you some more information about the project. Project structure — this folder is the core/purpose of the repo micrograd/ — the scalar class that supports basic math operations like , , , , , , and has a method that calculates a derivative of the expression, which is required for back-propagation flow. engine.ts Value add sub div mul pow exp tanh backward() nn.ts — the , , and (multi-layer perceptron) classes that implement a neural network on top of the differentiable scalar . Neuron Layer MLP Values - demo React application to experiment with the micrograd code demo/ src/demos/ - several playgrounds where you can experiment with the , , and classes. Neuron Layer MLP Micrograd See  🎬 The spelled-out intro to neural networks and back-propagation: building micrograd YouTube video (shared above) for a detailed explanation of how neural networks and backpropagation work. The video also explains in detail what the , , , and classes do. Neuron Layer MLP Value Briefly, the class allows you to build a computation graph for some expression that consists of scalar values. Value Here is an example of how the computation graph for the expression looks like: a * b + c Based on the class, we can build a expression . Here we're simulating a dot-product of matrix (input features) and matrix (neuron weights): Value Neuron X * W + b X W Out of , we can build the network class that consists of several of . The computation graph in this case may look a bit complex to be displayed here, but a simplified version might look like this: Neurons MLP Layers Neurons The main idea is that the computation graphs above "know" how to do automatic back propagation (in other words, how to calculate derivatives). This allows us to train the MLP network for several epochs and adjust the network weights in a way that reduces the ultimate loss: Demo (online) To see the online demo/playground, check the following link: 🔗 trekhleb.dev/micrograd-ts Demo (local) If you want to experiment with the code locally, follow the instructions below. Setup Clone the current repo locally. Switch to the demo folder: cd ./demo Setup node v18 using (optional): nvm nvm use Install dependencies: npm i Launch demo app: npm run dev The demo app will be available at http://localhost:5173/micrograd-ts Playgrounds Go to the ./demo/src/demos/ to explore several playgrounds for the , , and classes. Neuron Layer MLP I hope, playing around with the micrograd-ts code above and watching the video from Karpathy will be helpful at least for some of you, learners. Also published here.

Getting Started with Micrograd TS

Patterns and Practices for using SQLAlchemy 2.0 with FastAPI

Patrones y prácticas para usar SQLAlchemy 2.0 con FastAPI

Modèles et pratiques d'utilisation de SQLAlchemy 2.0 avec FastAPI

फास्टएपीआई के साथ SQLAlchemy 2.0 का उपयोग करने के लिए पैटर्न और अभ्यास

FastAPI で SQLAlchemy 2.0 を使用するためのパターンと実践

Padrões e práticas para usar SQLAlchemy 2.0 com FastAPI

Các mẫu và phương pháp sử dụng SQLAlchemy 2.0 với FastAPI

将 SQLAlchemy 2.0 与 FastAPI 结合使用的模式和实践

A couple of years ago, was all the rage, but it had one big issue: reactive stopped as soon as you accessed a SQL database. You had a nice reactive chain up to the database, defeating the whole purpose. Given the prevalence of SQL databases in existing and new apps, one couldn't enjoy the full benefits of Reactive Programming but still pay the full price of complexity. Reactive Programming Since then, the landscape has changed tremendously. Most importantly, it offers many reactive drivers over popular databases: PostgreSQL, MariaDB and MySQL, Microsoft SQL Server, Oracle, you name it! Even better, some frameworks provide a reactive API over them. Even though I'm not providing consulting services regularly, I wanted to keep up-to-date on accessing data reactively. In this post, I'll describe Hibernate Reactive, Spring Data R2DBC, and jOOQ in no particular order. The base application uses Project Reactor and its types - and . For an added twist, I use Kotlin (without coroutines). Most code snippets have unnecessary type hints for better understanding. Flux Mono The demo model I don't want a complicated demo model, but I don't want it to be too simple. I'll use a single many-to-many relationship and a field with : LocalDate Spring Data R2DBC As far as I remember, the Spring ecosystem was the first to offer a reactive database access API. At first, it was limited to H2 - not very useful in production. However, new reactive drivers were easy to integrate. Spring Data RDBC builds upon the widespread Spring Data JPA. The biggest difference is that there's a single required annotation for entities, . @Id Here's the code for the table: person data class Person(
    @Id val id: Long,
    val firstName: String,
    val lastName: String,
    val birthdate: LocalDate?,
    @Transient
    val addresses: MutableSet<Address> = mutableSetOf()
)

interface PersonRepository : ReactiveCrudRepository<Person, Long> R2DBC repositories look similar to regular Spring Data repositories with one big difference. They integrate Project Reactor's reactive types, and . Note that it's easy to use Kotlin's coroutines with an additional bridge dependency. Mono Flux Now comes the hard problem: mapping the many-to-many relationship with the . Address First, we must tell Spring Data R2DBC to use a specific constructor with an empty set of addresses. data class Person(
    @Id val id: Long,
    val firstName: String,
    val lastName: String,
    val birthdate: LocalDate?,
    @Transient
    val addresses: MutableSet<Address> = mutableSetOf()
) {
    @PersistenceCreator
    constructor(
        id: Long,
        firstName: String,
        lastName: String,
        birthdate: LocalDate? = null
    ) : this(id, firstName, lastName, birthdate, mutableSetOf())
} We also need to define the repository, as well as a query to list all addresses of a person: Address interface AddressRepository : ReactiveCrudRepository<Address, Long> {

    @Query("SELECT * FROM ADDRESS WHERE ID IN (SELECT ADDRESS_ID FROM PERSON_ADDRESS WHERE PERSON_ID = :id)")
    fun findAddressForPersonById(id: Long): Flux<Address>
} Now comes the least tasteful part: Spring Data R2DBC doesn't support many-to-many relationships at the moment. We need a hook that queries the addresses after loading a person. class PersonLoadOfficeListener(@Lazy private val repo: AddressRepository)   //1
  : AfterConvertCallback<Person> {

  override fun onAfterConvert(person: Person, table: SqlIdentifier) =
    repo.findAddressForPersonById(person.id)                                //2
      .mapNotNull {
          person.addresses.add(it)                                          //3
          person
      }.takeLast(1)                                                         //4
      .single(person)                                                       //5
} Annotate with to avoid running into circular dependencies exception during injection @Lazy Use the above query Add each address Reactive trick to wait for the last bit of data Turn into a single Person As far as I can understand, Spring Data R2DBC still needs to execute additional queries, thus leading to the (in)famous . N+1 query problem One configures database access via all available Spring alternatives: properties, YAML, Spring profiles, environment variables, etc. Here's a YAML example: spring.r2dbc:
  url: r2dbc:postgresql://localhost:5432/postgres?currentSchema=people
  username: postgres
  password: root Hibernate Reactive If you're familiar with regular Hibernate, you'll feel right at home with Hibernate Reactive. The mapping is the same in both cases: @Entity
@Table(name = "person", schema = "people")                           //1
class Person(
    @Id var id: Long?,
    @Column(name = "first_name")                                     //2
    var firstName: String?,
    @Column(name = "last_name")                                      //2
    var lastName: String?,
    var birthdate: LocalDate?,
    @ManyToMany
    @JoinTable(                                                      //3
        name = "person_address",
        schema = "people",
        joinColumns = [ JoinColumn(name = "person_id") ],
        inverseJoinColumns = [ JoinColumn(name = "address_id") ]
    )
    val addresses: MutableSet<Address> = mutableSetOf()
) {
    internal constructor() : this(null, null, null, null)            //4
} Define the table and the schema if necessary Define column names, if necessary Define the join column JPA requires a no-argument constructor We also need to configure the database. Hibernate Reactive uses the traditional XML-based JPA approach: <persistence xmlns="https://jakarta.ee/xml/ns/persistence"
           xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
           xsi:schemaLocation="https://jakarta.ee/xml/ns/persistence https://jakarta.ee/xml/ns/persistence/persistence_3_0.xsd"
           version="3.0">
  <persistence-unit name="postgresql">
    <provider>org.hibernate.reactive.provider.ReactivePersistenceProvider</provider>   
    <properties>
      <property name="jakarta.persistence.jdbc.url" value="jdbc:postgresql://localhost:5432/postgres?currentSchema=people" />
      <property name="jakarta.persistence.jdbc.user" value="postgres" />
      <property name="jakarta.persistence.jdbc.password" value="root" />
      <property name="jakarta.persistence.schema-generation.database.action" value="validate" />
    </properties>
  </persistence-unit>
</persistence> The only difference so far from the regular Hibernate configuration Here's the source for the query itself: val emf = Persistence.createEntityManagerFactory("postgresql")                            //1
val sessionFactory: Mutiny.SessionFactory = emf.unwrap(Mutiny.SessionFactory::class.java) //2
val people: Mono<MutableList<Person>> = sessionFactory
        .withSession {
            it.createQuery<Person>("SELECT p FROM Person p LEFT JOIN FETCH p.addresses a").resultList
        }.convert().with(UniReactorConverters.toMono())                                   //3 Regular EntityManagerFactory Unwrap the underlying session factory implementation. Because we configured a in the , it's a ReactivePersistenceProvider persistence.xml Mutiny.SessionFactory Hibernate Reactive integrates with , but an extension allows to bridge to Project Reactor if wanted Vert.x Note that Hibernate Reactive is the only library among the three to return a instead of a . In layman's terms, it means you get the whole list at once instead of getting the elements one by one and being able to do something on each one individually. Mono<List<T>> Flux<T> jOOQ Reactive As for the two above frameworks, jOOQ Reactive is similar to its non-reactive version. You first generate the code from the database schema, then use it. <plugin>
    <groupId>org.jooq</groupId>
    <artifactId>jooq-codegen-maven</artifactId>                               
    <executions>
        <execution>
            <id>jooq-codegen</id>
            <phase>generate-sources</phase>
            <goals>
                <goal>generate</goal>
            </goals>
        </execution>
    </executions>
    <dependencies>
        <dependency>
            <groupId>org.postgresql</groupId>                                 
            <artifactId>postgresql</artifactId>
            <version>42.6.0</version>
        </dependency>
    </dependencies>
    <configuration>
        <generator>
            <name>org.jooq.codegen.KotlinGenerator</name>                     
            <database>
                <inputSchema>people</inputSchema>                             
            </database>
            <target>
                <packageName>ch.frankel.blog.reactivedata.jooq</packageName>
            </target>
        </generator>
        <jdbc>                                                                
            <driver>org.postgresql.Driver</driver>
            <url>jdbc:postgresql://localhost:5432/postgres</url>
            <user>postgres</user>
            <password>root</password>
        </jdbc>
    </configuration>
</plugin> The version is defined in the parent Spring Boot Starter parent POM Set the necessary database driver(s). Note that one should use the driver non-reactive There's a Kotlin generator! Configure database configuration parameters Once you've generated the code, you can create your data class and design the query. jOOQ class hierarchy integrates with Java's collections, Java's Reactive Streams, and Project Reactor. The code may look complex if you're neither a SQL nor a jOOQ expert. Remember that variable types are unnecessary, but added for documentation purposes: fun findAll(): Flux<PersonWithAddresses> {                       //1
  val people: SelectJoinStep<Record5<Long?, String?, String?, LocalDate?, MutableList<Address>>> = //2
    ctx.select(
      PERSON.ID,
      PERSON.FIRST_NAME,
      PERSON.LAST_NAME,
      PERSON.BIRTHDATE,
      DSL.multiset(                                              //2
        DSL.select(
          PERSON_ADDRESS.ADDRESS_ID,
          PERSON_ADDRESS.address.FIRST_LINE,
          PERSON_ADDRESS.address.SECOND_LINE,
          PERSON_ADDRESS.address.ZIP,
          PERSON_ADDRESS.address.CITY,
          PERSON_ADDRESS.address.STATE,
          PERSON_ADDRESS.address.COUNTRY,
        ).from(PERSON_ADDRESS)
           .where(PERSON_ADDRESS.PERSON_ID.eq(PERSON.ID))
      ).convertFrom { it.map(addressMapper) }                   //3
  ).from(PERSON)
  return Flux.from(people)                                      //4
             .map(personWithAddressesMapper)                    //3
} Return a regular Project Reactor's Flux Use , see below. multiset Convert the row to an ordinary Java object via a function The magic happens here: wrap the regular query in a for is a Project Reactor Flux people Publisher Let's dive a bit into from the point of view of a non-jOOQ expert - me. Initially, I tried to execute a regular SQL query with results I tried to flatten with Project Reactor's API. I failed miserably because of my lack of knowledge of the API, but even if I had succeeded, it would have been the wrong approach. multiset After hours of research, I found via : multiset a post from Vlad Mihalcea The value constructor is one of jOOQ's and standard SQL's most powerful features. It allows for collecting the results of a into a single nested collection value with semantics. MULTISET non scalar subquery MULTISET -- MULTISET value constructor In the above query, we first select all addresses of a person, map each row to an object, and flatten them in a list on the same result row as the person. The second mapper maps the row, including the address list, to a dedicated person with an addresses list. I'm not a SQL master, so is hard at first glance. However, I confirm that it's a powerful feature indeed. multiset Note that . nested collections are fetched eagerly on a per-record basis, whereas top-level records are streamed reactively Conclusion We have browsed the surface of the main three reactive database access: Spring Data R2DBC, Hibernate, and jOOQ. So, which one should one choose? The main deciding factor is whether you already use one of their non-reactive flavors. Use the framework you're familiar with since both reactive and non-reactive usages are similar. I think that jOOQ is extremely powerful but requires a familiarity I currently lack. If you have complex queries that don't map easily to other approaches, it's the way to go. Besides that, I've no strong opinion, though I find Hibernate Reactive's configuration too limited by JPA and its return type puzzling. Mono<List<T>> Thanks for and for their reviews on their respective sections of expertise. Lukas Eder Mark Paluch The complete source code for this post can be found on . GitHub To go further: Hibernate Reactive Reference Documentation Integrating Hibernate Reactive with Spring jOOQ Reactive Fetching How to fetch multiple to-many relationships with jOOQ MULTISET How to Turn a List of Flat Elements into a Hierarchy in Java, SQL, or jOOQ Spring Data R2DBC Comment bien s'entendre avec avec Spring Data R2DBC... ou pas Originally at on July 9th, 2023 published A Java Geek

Reactive Programming - The New Way to Write Apps?

Part forty is here!!! It smells because there are likely many instances where it could be edited or improved. Most of these smells are just hints of something that might be wrong. Therefore, they are not required to be fixed per se… (You should look into it, though.) Previous Code Smells You can find all the previous code smells (Part i - ) . XXXIX here Let's continue... Code Smell 196 - Javascript Array Constructors Array creation is homogenous and predictable, or is it a hack? TL;DR: Be very careful with Javascript Arrays. Problems The Least surprise principle violation Solutions Be as declarative as possible Avoid creating Arrays with one argument. Context Javascript has a lot of magic tricks. Knowing them makes some developers proud and a sense of belonging to juniors. A language should be intuitive, homogeneous, predictable, simple, and pure. Sample Code Wrong const arrayWithFixedLength = new Array(3);

console.log(arrayWithFixedLength); // [ <5 empty items> ]
console.log(arrayWithFixedLength[0]); // Undefined
console.log(arrayWithFixedLength[1]); // Undefined
console.log(arrayWithFixedLength[3]); // Undefined
console.log(arrayWithFixedLength[4]); // Undefined
console.log(arrayWithFixedLength.length); // 3 Right const arrayWithTwoElements = new Array(3, 1);

console.log(arrayWithTwoElements); // [ 3, 1 ]
console.log(arrayWithTwoElements[0]); // 3
console.log(arrayWithTwoElements[1]); // 1
console.log(arrayWithTwoElements[2]); // Undefined
console.log(arrayWithTwoElements[5]); // Undefined
console.log(arrayWithTwoElements.length); // 2

const arrayWithTwoElementsLiteral = [3,1];

console.log(arrayWithTwoElementsLiteral); // [ 3, 1 ]
console.log(arrayWithTwoElementsLiteral[0]); // 3
console.log(arrayWithTwoElementsLiteral[1]); // 1
console.log(arrayWithTwoElementsLiteral[2]); // Undefined
console.log(arrayWithTwoElementsLiteral[5]); // Undefined
console.log(arrayWithTwoElementsLiteral.length); // 2 Detection Automatic [x] We can check for the notation with one argument and flag it as a warning. Tags Smart Conclusion Many "modern" languages are full of hacks to make life easier for programmers but are a source of potential undiscovered bugs. Relations Code Smell 06 - Too Clever Programmer Code Smell 69 - Big Bang (JavaScript Ridiculous Castings) Code Smell 93 - Send me Anything Disclaimer Code Smells are just my . opinion Credits Photo by on Ryan Quintal Unsplash Originally on this tweet: Twitter When someone says, "I want a programming language in which I need only say what I want done," give him a lollipop. Alan J. Perlis Software Engineering Great Quotes Code Smell 197 - Gratuitous Context TL;DR: Don't prefix or suffix your names with irrelevant information Problems Lack of Polymorphism Bad Naming violation with names Bijection Solutions Remove this context from your names Context In software development, gratuitous context refers to the unnecessary inclusion of additional information or data in code or user interfaces that do not contribute to the functionality or usability of the software. It can make the software more difficult to use, understand and maintain. It also increases the risk of errors or defects. Sample Code Wrong struct WEBBExoplanet {
    name: String,
    mass: f64, 
    radius: f64, 
    distance: f64, 
    orbital_period: f64, 
}

struct WEBBGalaxy {
    name: String,
    classification: String,
    distance: f64, 
    age: f64,
} Right struct Exoplanet {
    name: String,
    mass: f64, 
    radius: f64, 
    distance: f64, 
    orbital_period: f64, 
}

struct Galaxy {
    name: String,
    classification: String,
    distance: f64, 
    age: f64,
} Detection Semi-Automatic [x] We can find command patterns and rename all objects. Tags Naming Conclusion Class Preffixing was a widespread practice decades ago to claim ownership. Carefully consider the context and content of the software, and avoid including unnecessary or extraneous information wherever possible. Now we know clean names are more important. Relations Code Smell 141 - IEngine , AVehicle, ImplCar Code Smell 130 - AddressImpl Code Smell 174 - Class Name in Attributes More Info What exactly is a name - Part II Rehab Disclaimer Code Smells are my . opinion Credits Photo by on Mitchell Griest Unsplash The most dangerous kind of waste is the waste we do not recognize. Shigeo Shingo Code Smell 198 - Hidden Assumptions Software is about contracts and ambiguous contracts are a nightmare TL;DR: Keep your code explicit Problems Coupling Principle violation Fail Fast Fault Bijection Solutions Be declarative and explicit Don't oversimplify Context Hidden assumptions are underlying beliefs or expectations not explicitly stated in the code. They are still present and can impact the behavior of the software. Various reasons can give rise to assumptions such as incomplete requirements, incorrect presumptions about the user or environment, limitations of the programming language or tools, and bad accidental decisions. Sample Code Wrong tenCentimeters = 10
tenInches = 10

tenCentimeters + tenInches
# 20
# this error is based on the hidden assumption of a unit (any)
# and caused the Mars Climate Orbiter failure Right class Unit:
    def __init__(self, name, symbol):
        self.name = name
        self.symbol = symbol

class Measure:
    def __init__(self, scalar, unit):
        self.scalar = scalar
        self.unit = unit

    def __str__(self):
        return f"{self.scalar} {self.unit.symbol}"

centimetersUnit = Unit("centimeters", "cm")
inchesUnit = Unit("inches", "in")

tencCentimeters = Measure(10, centimeters)
tenInches = Measure(10, inches)

tenCentimeters + tenInches
# error until we introduce a conversion factor
# in this case the conversion is constant 
# inches = centimeters / 2.54 Detection Manual [x] This is a design smell Tags Coupling Conclusion Hidden assumptions can be difficult to identify and can lead to bugs, security vulnerabilities, and usability issues. To mitigate these risks, software developers should be aware of their assumptions and biases. Developers also need to engage with users to understand their needs and expectations. They must test their software in various scenarios to uncover hidden assumptions and edge cases. Relations Code Smell 02 - Constants and Magic Numbers More Info Mars Climate Orbiter Disaster Coupling - The one and only software design problem Measure Solution Disclaimer Code Smells are my . opinion Credits Photo by on Christian Pfeifer Unsplash A human organization is just as much an information system as any computer system. It is almost certainly more complex, but the same fundamental ideas apply. Things that are fundamentally difficult, like concurrency and coupling, are difficult in the real world of people, too. Dave Farley Code Smell 199 - Gratuitous Booleans A code that either returns or no returns at all TL;DR: Check carefully your boolean expressions Problems Readability Possible Defects Solutions Refactor and remove obsolete code Context When a function is designed to return an invariant value, it may be poor design, but it shouldn’t adversely affect the outcome of your program. However, when it happens on all paths through the logic, it is likely a mistake. This rule raises an issue when a function contains several return statements that all return the same value. Sample Code Wrong # Gratuitous boolean expressions

if a > 0 and True:
    print("a is positive")
else:
    print("a is not positive") Right if a > 0:
    print("a is positive")
else:
    print("a is not positive") Detection Automatic [x] Many can detect this problem by parsing execution trees. linters Tags Complexity Conclusion Boolean expressions should be straightforward to read and understand. Relations Code Smell 115 - Return True Code Smell 118 - Return False Code Smell 101 - Comparison Against Booleans More Info How to Get Rid of Annoying IFs Forever SonarSource Disclaimer Code Smells are just my . opinion Credits Photo by on Jungwoo Hong Unsplash The central enemy of reliability is complexity. Daniel Geer Code Smell 200 - Poltergeist An object that appears and disappears mysteriously TL;DR: Add the necessary indirection layers, but no more. Problems complexity Accidental Readability YAGNI violation Solutions Remove the intermediate object Context A poltergeist (or gypsy wagon) is a short-lived object used to perform initialization or to invoke methods in another, more permanent class. An object is responsible for many small tasks, resulting in excessive coupling and a lack of cohesion in the code. Sample Code Wrong public class Driver
{
    private Car car;

    public Driver(Car car)
    {
        this.car = car;
    }

    public void DriveCar()
    {
        car.driveCar();
    }
}

Car porsche = new Car();
Driver homer = new Driver(porsche);
homer.DriveCar(); Right Car porsche = new Car();

porsche.driveCar();
// We don't need the driver Detection Manual [x] This is a design smell. Tags Complexity Conclusion Don't add accidental complexity to the essential complexity we already have. Remove objects if they are not needed. middleman Relations Code Smell 54 - Anchor Boats Code Smell 67 - Middle Man More Info Wikipedia Disclaimer Code Smells are my . opinion Credits Photo by on Lan Gao Unsplash The art of programming is the art of organizing complexity, of mastering multitude and avoiding its bastard chaos as effectively as possible. E. W. Dijkstra 5 more code smells are coming soon…

How to Find the Stinky Parts of Your Code [Part XL]

As a researcher, you're often up against complex, heavily obfuscated samples. And GuLoader, the malware we're dealing with today, is a classic case in point. malware Take a look at this pseudo-code, produced by decompiling its assembly code — it is ugly and unreadable. Faced with a puzzle like this, you might find yourself at a loss. Where should you even begin? How do you tackle the analysis of this sample? Let's break it down. In this article, then, we’ll explore strategies to deobfuscate such code, using GuLoader as a reference. You’ll learn about: Common obfuscation tactics used by adversaries How to defeat them How to deobfuscate a GuLoader sample The article is based on the previously published by ANY.RUN. Visit our blog to find the sample we'll analyze, as well as unpacking instructions and a Ghidra script that partially automates much of what we’re going to cover from this point on. GuLoader malware analysis This article focuses on static analysis. But if you want to analyze a GuLoader sample dynamically, you can use cloud malware sandbox. ANY.RUN of the Enterprise plan using your business email. Take advantage of the extended lifetime of VM instances, unlimited tasks, and Windows versions 7 through 11. Apply for a free 14-day trial Dynamic analysis lets you see how malware works in the real world by linking its behavior to its technical setup. It's a way to check its actions across different system setups and collect IOCs. Without delaying further, let's dive in. Identifying Obfuscation Methods After unpacking the sample, we begin with a manual analysis of the shellcode and quickly realize that it’s obfuscated. Studying the code enables us to group the obfuscation techniques GuLoader employs into categories: XMM instructions Unconditional JMP instructions Junk instructions Fake comparison instructions Fake PUSHAD instructions Fake PUSH instructions Opaque predicates And arithmetic expressions This is a good time to stop, analyze the obfuscation methods being used, and develop a debofuscation strategy. For example, in our case, most of these techniques introduce code that doesn't alter the final execution outcome. Therefore, we can often safely “NOP” them to improve readability. But proceed with caution — not all obfuscated code is irrelevant to the program's operation, as we’ll soon discover. Now, let's examine these obfuscation techniques individually, and see how to defeat them. 1. XMM instructions The code is littered with many XMM instructions. These appear disordered and add complexity to the analysis process. We encounter them right from the first byte of the unpacked shellcode. Note that many emulation engines stumble over them due to the lack of default support. We put Angr, Triton, and Ghidra's embedded engines to the test — all fell short. How we dealt with it In GuLoader's case, XMM instructions don't actually impact the intended behavior of the code. You’ll encounter similar obfuscation methods in a lot of malware. Consequently, we can safely replace all XMM instructions with "NOP," as shown in the following table: Here's how the result looks in the Ghidra disassembler: 2. Unconditional JMP instructions Unconditional JMP instructions segment the code into smaller chunks. This method is frequently used to avoid detection by antivirus software and other security tools. Additionally, it can make the analysts' job more time-consuming and frustrating, as they must jump between these blocks, especially when dealing with a large amount of code. GuLoader and other malware commonly employ this technique. How we dealt with it This obfuscation method is quite easy to defeat. The disassembler in the decompiled code often successfully concatenates these blocks, improving code readability, even with these unconditional jumps present. As such, we can leave the small blocks as they are without needing to merge them. 3. Junk instructions Junk assembly instructions often come into play as an extra layer of obfuscation. These instructions do not perform any tangible function, typically leaving the register values, execution flow, or memory unaltered. You will encounter these within GuLoader as well. Look out for instructions that perform no action ("NOP", "FNOP") and those that shift or rotate by zero bits ("SHL reg, 0"; "ROL reg, 0"). Other non-impactful instructions like "OR reg, 0", "XOR reg, 0", "CLD", "WAIT" are also present. How we dealt with it Addressing fake comparison instructions is more challenging than just replacing junk ones with "NOP". We can't remove all comparison instructions as some are necessary for the correct code function. One way to tackle this is to “mark” all comparison instructions we encounter. If no instructions are found that use the comparison's result, it's safe to NOP it. If we find a conditional jump or similar, we unmark the comparison to avoid removal. The following table shows an example where all comparison instructions except "CMP EDX,0x0" were selectively replaced with NOP: 5. Fake PUSHAD instructions GuLoader also employs the obfuscation tactic of using fake "PUSHAD" instructions, coupled with a matching "POPAD" instruction. They can temporarily modify register values but are nullified by the "POPAD" restoring the original register values. How we dealt with it Our research showed that all "PUSHAD" instructions in GuLoader are extraneous. So, we address this by replacing "PUSHAD", "POPAD", and the intermediate instructions with NOP: However, not all "POPAD" instructions in GuLoader are junk. We leave those without a corresponding "PUSHAD" untouched. 6. Fake PUSH instructions Another obfuscation technique similar to the previous one is the use of fake "PUSH" instructions. These instructions push a value onto the stack only to pop it off immediately. An example is the inclusion of a "PUSH SS" instruction, possibly followed by instructions modifying a register or memory location. However, the subsequent "POP SS" restores the stack pointer to its initial value. How we dealt with it Defeating fake PUSH instructions resembles the process for fake PUSHAD, but it's crucial to leave the non-pushed registers unaltered. 7. Opaque predicates Opaque predicates are conditional statements that always return true or false, yet they're tough to analyze or predict. These are found in GuLoader's code and complicate the logic understanding. For instance, a pair of instructions like “MOV BL, 0xB6” and “CMP BL, 0xB6” could be followed by a conditional jump like “JNZ ADDR”. The comparison always returns false since the compared value equals the moved value, making the jump unnecessary and baffling. How we dealt with it Overcoming opaque predicates can seem challenging due to the requirement of "predicting" the jump condition. However, our situation is more straightforward as all opaque predicates fall within the "PUSHAD" and "POPAD" blocks. Therefore, we simply replace all predicates between these instructions with NOP. 8. Arithmetic Expressions Obfuscated arithmetic expressions are among the more interesting techniques used by GuLoader. They make it tougher to understand the actual operations performed. These expressions incorporate arithmetic actions like addition or subtraction. Sometimes, they’re mixed with other obfuscations like fake comparisons, opaque predicates, and junk instructions. One example is moving a constant value into a register and executing arithmetic operations: Another instance is pushing a constant value onto the stack and performing calculations on the memory: How we dealt with it To deobfuscate GuLoader's arithmetic expressions, we adopt an approach similar to handling fake comparisons. We mark all "MOV" instructions where the second argument is a scalar value, and all "PUSH" instructions where the argument is scalar. On encountering an arithmetic operation, we update the constant value in the first instruction and replace the current one with NOP. This way, the first instruction has the result, and subsequent arithmetic instructions are replaced by NOP. Below are examples with optimized “MOV” and “PUSH” operations: Be cautious with operand sizes. Preserving the correct size during operations is crucial. Wrapping up our analysis In this article from experts, we used GuLoader as a real-world example to highlight typical deobfuscation strategies. We started with analyzing the code, then grouped similar obfuscation tactics, and finally, thought out unique ways to tackle each. ANY.RUN But it's crucial to understand that these techniques aren't a one-size-fits-all solution. Each malware sample may present its unique obfuscation strategies that need unique countermeasures. Our discussion underlines the significance of breaking down a complex task like deobfuscation into clear, manageable steps. Remember, that successful deobfuscating of any malware involves detailed analysis, spotting obfuscation patterns, and creating optimized strategies.

Scalar

scalar-labs.com

6 employees

Since 2017

COMPANY RANKING

Scalar

EVERGREEN INDEX #2895

See more notable links about Scalar

Apply to get your Tech Company News Page live on HackerNoon.com today!

Read More Tech Stories Related to #Scalar

Scalar

Scalar

scalar-labs.com

6 employees

Since 2017

COMPANY RANKING

Scalar

EVERGREEN INDEX #2895

Related CompaniesRank

Scalar Story MentionsComplete List

See more notable links about Scalar

Apply to get your Tech Company News Page live on HackerNoon.com today!

Read More Tech Stories Related to #Scalar

Scalar