NotCommon

notcommon

Understanding the Twitter API So You Can Design Your Own

Разуменне Twitter API, каб вы маглі стварыць свой уласны

Κατανόηση του API του Twitter ώστε να μπορείτε να σχεδιάσετε το δικό σας

Entender la API de Twitter para que puedas diseñar la tuya propia

Twitter APIa ulertzea, zurea diseinatu ahal izateko

Comprender a API de Twitter para que poidas deseñar a túa propia

Հասկանալով Twitter API-ն, որպեսզի կարողանաք նախագծել ձեր սեփականը

Memahami API Twitter Sehingga Anda Dapat Mendesain Sendiri

הבנת ממשק API של Twitter כך שתוכל לעצב בעצמך

Twitter APIを理解して独自のAPIを設計する

Pochopenie rozhrania Twitter API, aby ste si mohli navrhnúť svoj vlastný

Kuptimi i API-së së Twitter në mënyrë që të mund të dizajnoni tuajën

Twitter API-ä düşünmek, şonuň üçin özüňiz düzüp bilersiňiz

Author:
(1) David Staines. Table of Links Abstract 1 Introduction 2 Mathematical Arguments 3 Outline and Preview 4 Calvo Framework and 4.1 Household’s Problem 4.2 Preferences 4.3 Household Equilibrium Conditions 4.4 Price-Setting Problem 4.5 Nominal Equilibrium Conditions 4.6 Real Equilibrium Conditions and 4.7 Shocks 4.8 Recursive Equilibrium 5 Existing Solutions 5.1 Singular Phillips Curve 5.2 Persistence and Policy Puzzles 5.3 Two Comparison Models 5.4 Lucas Critique 6 Stochastic Equilibrium and 6.1 Ergodic Theory and Random Dynamical Systems 6.2 Equilibrium Construction 6.3 Literature Comparison 6.4 Equilibrium Analysis 7 General Linearized Phillips Curve 7.1 Slope Coefficients 7.2 Error Coefficients 8 Existence Results and 8.1 Main Results 8.2 Key Proofs 8.3 Discussion 9 Bifurcation Analysis 9.1 Analytic Aspects 9.2 Algebraic Aspects (I) Singularities and Covers 9.3 Algebraic Aspects (II) Homology 9.4 Algebraic Aspects (III) Schemes 9.5 Wider Economic Interpretations 10 Econometric and Theoretical Implications and 10.1 Identification and Trade-offs 10.2 Econometric Duality 10.3 Coefficient Properties 10.4 Microeconomic Interpretation 11 Policy Rule 12 Conclusions and References Appendices A Proof of Theorem 2 and A.1 Proof of Part (i) A.2 Behaviour of ∆ A.3 Proof Part (iii) B Proofs from Section 4 and B.1 Individual Product Demand (4.2) B.2 Flexible Price Equilibrium and ZINSS (4.4) B.3 Price Dispersion (4.5) B.4 Cost Minimization (4.6) and (10.4) B.5 Consolidation (4.8) C Proofs from Section 5, and C.1 Puzzles, Policy and Persistence C.2 Extending No Persistence D Stochastic Equilibrium and D.1 Non-Stochastic Equilibrium D.2 Profits and Long-Run Growth E Slopes and Eigenvalues and E.1 Slope Coefficients E.2 Linearized DSGE Solution E.3 Eigenvalue Conditions E.4 Rouche’s Theorem Conditions F Abstract Algebra and F.1 Homology Groups F.2 Basic Categories F.3 De Rham Cohomology F.4 Marginal Costs and Inflation G Further Keynesian Models and G.1 Taylor Pricing G.2 Calvo Wage Phillips Curve G.3 Unconventional Policy Settings H Empirical Robustness and H.1 Parameter Selection H.2 Phillips Curve I Additional Evidence and I.1 Other Structural Parameters I.2 Lucas Critique I.3 Trend Inflation Volatility 5.3 Two Comparison Models This subsection introduces two alternative models featuring monetary Non neutrality. They will be used in the next section and throughout the paper as a point of comparison with solutions of the Calvo model. The first model is classical, in the sense that it features perfectly flexible prices. The second is Keynesian because prices are not able to adjust fully to inflation. Along with the menu cost model discussed in Section 8, it is an example of a state-dependent pricing model. Crucially, it does not feature staggered optimization. Prices will typically change in every period. I layout the equivalence between the existing solution of Calvo and the Rotemberg model, I extend this equivalence to the original Lucas model, allowing me to view it as part of the observational equivalence claim of Lucas Jr [1976]. Finally, I briefly review the rest of the critique. 5.3.1 Lucas Model Lucas Jr [1972] is the original rational expectations monetary model. Markets are perfectly competitive and anticipated monetary shocks are impotent. Monetary non-neutrality arises because of imperfect information rather than sticky prices. Nevertheless, by adding a money demand function, I am able to derive surprising similarity to New Keynesian models with price rigidity. There are a collection of price-taking households.[34] Each produces a single good, using just its own labor.[35] The choice problem simplifies to a static labor-leisure trade-of The first order condition balances marginal revenue and marginal cost so Solving and log-linearizing gives an expression for an individual firm’s supply curve For parsimony, a reduced form aggregate demand curve replaces the Euler equation (18). Idiosyncratic taste shocks, reflected by the disturbance zt(i) in the individual firms demand schedule, are essential to the Lucas model. I assume they are independent and identically distributed across producers and time, with tail restrictions such that they will cancel out in aggregate almost surely. Here is the log-linear form There is linear aggregation so whilst Monetary non-neutrality is entirely driven by unexpected shocks. There is no role for price dispersion here, since the firms are price-takers and the distribution of relative price is unaffected by inflation. Unlike its Keynesian counterparts, the Lucas model would be unaffected by trend inflation. Using the individual supply curve (62), demand curve (64), steady state deviation (68) and the Lucas supply curve, (71) and (72) implies the expression for relative price deviation and from (69) and (70) This allows me to derive the following implicit formulation final form for the slope of the Lucas surprise Phillips curve 5.3.2 Rotemberg Pricing This model Rotemberg [1982] is perhaps the simplest formulation of state dependent pricing and the one most commonly compared to Calvo. The idea is that firms face a convex cost of changing prices and therefore they do not adjust immediately to their optimal flexible price. The problem is that convex costs of price changing imply firms change their prices every period through small adjustments. This runs contrary to an overwhelming body of evidence of infrequent large adjustment discussed in Appendix H.1.3. Therefore the first order conditions simplify to Finally, the resource constraint reflecting a wedge between production and consumption caused by the cost of price changing replaces (20) Author:
(1) David Staines. This paper is available on arxiv under CC 4.0 license. [34] It is not common to have differentiated products and price-taking behavior. For motivation imagine there were actually multiple households producing the same good and therefore competition between them or the possibility to set-up firms with a consequent free entry condition. Recall the Bertrand paradox that two competing firms can implement a competitive equilibrium. [35] This formulation comes from Romer [2012]. The idea is to avoid the households using the economy-wide labor market to deduce the aggregate price level. In the original paper, Lucas used an overlapping generations structure where households produced in one period then sold output in the next. Recent literature has developed more sophisticated stories about dispersed private information and strategic interaction. I do not pursue this approach as I am looking for a benchmark model. Author: (1) David Staines. Author: Author: (1) David Staines. Table of Links Abstract Abstract 1 Introduction 1 Introduction 2 Mathematical Arguments 2 Mathematical Arguments 3 Outline and Preview 3 Outline and Preview 4 Calvo Framework and 4.1 Household’s Problem 4 Calvo Framework and 4.1 Household’s Problem 4.2 Preferences 4.2 Preferences 4.3 Household Equilibrium Conditions 4.3 Household Equilibrium Conditions 4.4 Price-Setting Problem 4.4 Price-Setting Problem 4.5 Nominal Equilibrium Conditions 4.5 Nominal Equilibrium Conditions 4.6 Real Equilibrium Conditions and 4.7 Shocks 4.6 Real Equilibrium Conditions and 4.7 Shocks 4.8 Recursive Equilibrium 4.8 Recursive Equilibrium 5 Existing Solutions 5 Existing Solutions 5.1 Singular Phillips Curve 5.1 Singular Phillips Curve 5.2 Persistence and Policy Puzzles 5.2 Persistence and Policy Puzzles 5.3 Two Comparison Models 5.3 Two Comparison Models 5.4 Lucas Critique 5.4 Lucas Critique 6 Stochastic Equilibrium and 6.1 Ergodic Theory and Random Dynamical Systems 6 Stochastic Equilibrium and 6.1 Ergodic Theory and Random Dynamical Systems 6.2 Equilibrium Construction 6.2 Equilibrium Construction 6.3 Literature Comparison 6.3 Literature Comparison 6.4 Equilibrium Analysis 6.4 Equilibrium Analysis 7 General Linearized Phillips Curve 7 General Linearized Phillips Curve 7.1 Slope Coefficients 7.1 Slope Coefficients 7.2 Error Coefficients 7.2 Error Coefficients 8 Existence Results and 8.1 Main Results 8 Existence Results and 8.1 Main Results 8.2 Key Proofs 8.2 Key Proofs 8.3 Discussion 8.3 Discussion 9 Bifurcation Analysis 9 Bifurcation Analysis 9.1 Analytic Aspects 9.1 Analytic Aspects 9.2 Algebraic Aspects (I) Singularities and Covers 9.2 Algebraic Aspects (I) Singularities and Covers 9.3 Algebraic Aspects (II) Homology 9.3 Algebraic Aspects (II) Homology 9.4 Algebraic Aspects (III) Schemes 9.4 Algebraic Aspects (III) Schemes 9.5 Wider Economic Interpretations 9.5 Wider Economic Interpretations 10 Econometric and Theoretical Implications and 10.1 Identification and Trade-offs 10 Econometric and Theoretical Implications and 10.1 Identification and Trade-offs 10.2 Econometric Duality 10.2 Econometric Duality 10.3 Coefficient Properties 10.3 Coefficient Properties 10.4 Microeconomic Interpretation 10.4 Microeconomic Interpretation 11 Policy Rule 11 Policy Rule 12 Conclusions and References 12 Conclusions and References Appendices Appendices A Proof of Theorem 2 and A.1 Proof of Part (i) A Proof of Theorem 2 and A.1 Proof of Part (i) A.2 Behaviour of ∆ A.2 Behaviour of ∆ A.3 Proof Part (iii) A.3 Proof Part (iii) B Proofs from Section 4 and B.1 Individual Product Demand (4.2) B Proofs from Section 4 and B.1 Individual Product Demand (4.2) B.2 Flexible Price Equilibrium and ZINSS (4.4) B.2 Flexible Price Equilibrium and ZINSS (4.4) B.3 Price Dispersion (4.5) B.3 Price Dispersion (4.5) B.4 Cost Minimization (4.6) and (10.4) B.4 Cost Minimization (4.6) and (10.4) B.5 Consolidation (4.8) B.5 Consolidation (4.8) C Proofs from Section 5, and C.1 Puzzles, Policy and Persistence C Proofs from Section 5, and C.1 Puzzles, Policy and Persistence C.2 Extending No Persistence C.2 Extending No Persistence D Stochastic Equilibrium and D.1 Non-Stochastic Equilibrium D Stochastic Equilibrium and D.1 Non-Stochastic Equilibrium D.2 Profits and Long-Run Growth D.2 Profits and Long-Run Growth E Slopes and Eigenvalues and E.1 Slope Coefficients E Slopes and Eigenvalues and E.1 Slope Coefficients E.2 Linearized DSGE Solution E.2 Linearized DSGE Solution E.3 Eigenvalue Conditions E.3 Eigenvalue Conditions E.4 Rouche’s Theorem Conditions E.4 Rouche’s Theorem Conditions F Abstract Algebra and F.1 Homology Groups F Abstract Algebra and F.1 Homology Groups F.2 Basic Categories F.2 Basic Categories F.3 De Rham Cohomology F.3 De Rham Cohomology F.4 Marginal Costs and Inflation F.4 Marginal Costs and Inflation G Further Keynesian Models and G.1 Taylor Pricing G Further Keynesian Models and G.1 Taylor Pricing G.2 Calvo Wage Phillips Curve G.2 Calvo Wage Phillips Curve G.3 Unconventional Policy Settings G.3 Unconventional Policy Settings H Empirical Robustness and H.1 Parameter Selection H Empirical Robustness and H.1 Parameter Selection H.2 Phillips Curve H.2 Phillips Curve I Additional Evidence and I.1 Other Structural Parameters I Additional Evidence and I.1 Other Structural Parameters I.2 Lucas Critique I.2 Lucas Critique I.3 Trend Inflation Volatility I.3 Trend Inflation Volatility 5.3 Two Comparison Models This subsection introduces two alternative models featuring monetary Non neutrality. They will be used in the next section and throughout the paper as a point of comparison with solutions of the Calvo model. The first model is classical, in the sense that it features perfectly flexible prices. The second is Keynesian because prices are not able to adjust fully to inflation. Along with the menu cost model discussed in Section 8, it is an example of a state-dependent pricing model. Crucially, it does not feature staggered optimization. Prices will typically change in every period. I layout the equivalence between the existing solution of Calvo and the Rotemberg model, I extend this equivalence to the original Lucas model, allowing me to view it as part of the observational equivalence claim of Lucas Jr [1976]. Finally, I briefly review the rest of the critique. 5.3.1 Lucas Model Lucas Jr [1972] is the original rational expectations monetary model. Markets are perfectly competitive and anticipated monetary shocks are impotent. Monetary non-neutrality arises because of imperfect information rather than sticky prices. Nevertheless, by adding a money demand function, I am able to derive surprising similarity to New Keynesian models with price rigidity. There are a collection of price-taking households.[34] Each produces a single good, using just its own labor.[35] The choice problem simplifies to a static labor-leisure trade-of The first order condition balances marginal revenue and marginal cost so Solving and log-linearizing gives an expression for an individual firm’s supply curve For parsimony, a reduced form aggregate demand curve replaces the Euler equation (18). Idiosyncratic taste shocks, reflected by the disturbance zt(i) in the individual firms demand schedule, are essential to the Lucas model. I assume they are independent and identically distributed across producers and time, with tail restrictions such that they will cancel out in aggregate almost surely. Here is the log-linear form There is linear aggregation so whilst Monetary non-neutrality is entirely driven by unexpected shocks. There is no role for price dispersion here, since the firms are price-takers and the distribution of relative price is unaffected by inflation. Unlike its Keynesian counterparts, the Lucas model would be unaffected by trend inflation. Using the individual supply curve (62), demand curve (64), steady state deviation (68) and the Lucas supply curve, (71) and (72) implies the expression for relative price deviation and from (69) and (70) This allows me to derive the following implicit formulation final form for the slope of the Lucas surprise Phillips curve 5.3.2 Rotemberg Pricing This model Rotemberg [1982] is perhaps the simplest formulation of state dependent pricing and the one most commonly compared to Calvo. The idea is that firms face a convex cost of changing prices and therefore they do not adjust immediately to their optimal flexible price. The problem is that convex costs of price changing imply firms change their prices every period through small adjustments. This runs contrary to an overwhelming body of evidence of infrequent large adjustment discussed in Appendix H.1.3. Therefore the first order conditions simplify to Finally, the resource constraint reflecting a wedge between production and consumption caused by the cost of price changing replaces (20) Author: (1) David Staines. Author: Author: (1) David Staines. This paper is available on arxiv under CC 4.0 license. This paper is available on arxiv under CC 4.0 license. available on arxiv [34] It is not common to have differentiated products and price-taking behavior. For motivation imagine there were actually multiple households producing the same good and therefore competition between them or the possibility to set-up firms with a consequent free entry condition. Recall the Bertrand paradox that two competing firms can implement a competitive equilibrium. [35] This formulation comes from Romer [2012]. The idea is to avoid the households using the economy-wide labor market to deduce the aggregate price level. In the original paper, Lucas used an overlapping generations structure where households produced in one period then sold output in the next. Recent literature has developed more sophisticated stories about dispersed private information and strategic interaction. I do not pursue this approach as I am looking for a benchmark model.

Why Prices Don’t Change Overnight: Insights from Two Key Economic Models 

Disclaimer: As a frequent user of satellite images in my work (and not an expert in the field), I've often found myself with these kinds of doubts. The more I learned about the subject, the more I realized how little I actually knew. So, in an effort to use this type of data more effectively and help my colleagues with their frequent questions, I'm writing these notes. There are many materials that cover this topic in greater depth. When we start using satellite images in our analyses - and for many, their first contact with this type of data happens at work - it's common for someone with a bit more knowledge about the data to provide some key information to help you get started. One of these pieces of information is: The larger the pixel size, the lower the spatial resolution of the image, and vice versa. This association makes a lot of sense for understanding how "large" and "small" pixels can impact the analysis you want to perform. But if you start reading more about the subject with some questions, you'll find there are quite a few concepts behind it, and knowing a bit more about this could help you in the future. I'll try to make a brief connection between some of these concepts, clarifying why pixels and spatial resolution are not essentially synonymous, and addressing some common doubts I see in my day-to-day work. Key Concepts: Spatial resolution


Pixel size


Ground Sample Distance (GSD)


Ground Resolved Distance (GRD) It all starts with the satellite sensor in orbit and its detectors. Considering a single detector in the sensor, the angle through which it is sensitive to radiation is called IFOV (Instantaneous Field of View). The larger the IFOV, the larger the area, and the lower the ability to distinguish different objects, and vice versa. In the figure above, IFOV1 allows images with greater detail of objects compared to IFOV2, in their corresponding: GRC1 and GRC2 [Ground Resolved Cell] (associations of the satellite's flight height, H, with IFOV). The size of these cells is called Ground Resolved Distance, or GRD. The concept of GRD is also related to the design and size of the detectors. Looking at the figure, we can already understand that IFOV1, and consequently GRC1 and GRD1, have less spectral "mixing": perhaps a person and a bit of ground. In IFOV2, everything within its GRD is "mixed" and lacks the ability to distinguish different targets: person, ground, object. Here, the relationship between these concepts and spatial resolution becomes clearer. Ground Sample Distance, or GSD, is a concept related to the distance between cell centers on the surface, in summary. What's not yet clear is the association, not always correct, with spatial resolution. This occurs because in practice, GSD and GRD are not always exactly the same - there's some overlap between the IFOV of neighboring detectors (in an ideal scenario, this overlap would be minimal to non-existent, and characteristics like sensor quality and IFOV size influence this). What follows from this is that GRD is generally a better proxy for spatial resolution than GSD. During the radiometric signal acquisition, there's processing before the product is delivered. In this processing, in geometric terms, reprojection and resampling are common, so that products are organized into constant grids, independent of acquisition variations, such as angle. From the organization of data in these grids, we get the pixel size And that's why the pixel size will be, at most, equal to GSD. (And that's also why we often hear that a pixel corresponds to the distance between two consecutive centroids). IF, in an ideal scenario (where sensor quality is crucial), GSD is very similar to GRD (minimal overlap), then we could understand all these concepts as good proxies for spatial resolution. Why "could"? Because the acquired signal can go through a resampling process resulting in, say, a larger pixel size, and consequently lower spatial resolution. The opposite is not recommended. If you simply take a signal acquired with a certain GSD, GRD, and in the resampling process "build" a smaller pixel, in which you supposedly want more detailed information, this will not occur (Pixel size <= GSD!!). The signal was already acquired under certain conditions, and you can't change that (GRD) to "detail" the image more. In summary, as I've often had to understand in various ways, even to explain to colleagues: If we resample an image to a smaller pixel size, we are NOT improving spatial resolution! Another fact is that as users, we end up learning more about pixel size and spatial resolution of an image. It's not common for us to have access to (or seek out) information like GRD or GSD. There's much more to this, but this is a start to clarify the relationship between these concepts, and why pixel size does not necessarily correspond to the spatial resolution of an image! References: *reference 2 was an important reference for this text, for those who want to delve deeper. [1] Spatial Resolution, Pixel Size, and Scale [2] The most misunderstood words in Earth Observation [3] Remote Sensing INFORMATION SHEET Disclaimer: As a frequent user of satellite images in my work (and not an expert in the field), I've often found myself with these kinds of doubts. The more I learned about the subject, the more I realized how little I actually knew. So, in an effort to use this type of data more effectively and help my colleagues with their frequent questions, I'm writing these notes. There are many materials that cover this topic in greater depth. many materials When we start using satellite images in our analyses - and for many, their first contact with this type of data happens at work - it's common for someone with a bit more knowledge about the data to provide some key information to help you get started. One of these pieces of information is: The larger the pixel size, the lower the spatial resolution of the image, and vice versa. The larger the pixel size, the lower the spatial resolution of the image, and vice versa. The larger the pixel size, the lower the spatial resolution of the image, and vice versa. larger the pixel size, the lower the spatial resolution This association makes a lot of sense for understanding how "large" and "small" pixels can impact the analysis you want to perform. But if you start reading more about the subject with some questions, you'll find there are quite a few concepts behind it, and knowing a bit more about this could help you in the future. I'll try to make a brief connection between some of these concepts, clarifying why pixels and spatial resolution are not essentially synonymous , and addressing some common doubts I see in my day-to-day work. why pixels and spatial resolution are not essentially synonymous Key Concepts: Spatial resolution Pixel size Ground Sample Distance (GSD) Ground Resolved Distance (GRD) Spatial resolution Spatial resolution Pixel size Pixel size Ground Sample Distance (GSD) Ground Sample Distance (GSD) Ground Resolved Distance (GRD) Ground Resolved Distance (GRD) It all starts with the satellite sensor in orbit and its detectors. Considering a single detector in the sensor, the angle through which it is sensitive to radiation is called IFOV (Instantaneous Field of View). The larger the IFOV, the larger the area, and the lower the ability to distinguish different objects, and vice versa. IFOV larger the IFOV, the larger the area, and the lower the ability to distinguish In the figure above, IFOV1 allows images with greater detail of objects compared to IFOV2, in their corresponding: GRC1 and GRC2 [ Ground Resolved Cell ] (associations of the satellite's flight height, H, with IFOV). The size of these cells is called Ground Resolved Distance, or GRD . The concept of GRD is also related to the design and size of the detectors. Ground Resolved Cell GRD Looking at the figure, we can already understand that IFOV1, and consequently GRC1 and GRD1, have less spectral "mixing": perhaps a person and a bit of ground. In IFOV2, everything within its GRD is "mixed" and lacks the ability to distinguish different targets: person, ground, object. Here, the relationship between these concepts and spatial resolution becomes clearer. Ground Sample Distance, or GSD , is a concept related to the distance between cell centers on the surface, in summary. GSD What's not yet clear is the association, not always correct, with spatial resolution. This occurs because in practice, GSD and GRD are not always exactly the same - there's some overlap between the IFOV of neighboring detectors (in an ideal scenario, this overlap would be minimal to non-existent, and characteristics like sensor quality and IFOV size influence this). in practice, GSD and GRD are not always exactly the same overlap What follows from this is that GRD is generally a better proxy for spatial resolution than GSD. What follows from this is that GRD is generally a better proxy for spatial resolution than GSD. GRD is generally a better proxy for spatial resolution During the radiometric signal acquisition, there's processing before the product is delivered. In this processing, in geometric terms, reprojection and resampling are common, so that products are organized into constant grids, independent of acquisition variations, such as angle. From the organization of data in these grids, we get the pixel size From the organization of data in these grids, we get the pixel size From the organization of data in these grids, we get the pixel size organization of data in these grids, pixel size And that's why the pixel size will be, at most, equal to GSD . (And that's also why we often hear that a pixel corresponds to the distance between two consecutive centroids). pixel size will be, at most, equal to GSD IF, in an ideal scenario (where sensor quality is crucial), GSD is very similar to GRD (minimal overlap), then we could understand all these concepts as good proxies for spatial resolution. Why "could"? Because the acquired signal can go through a resampling process resulting in, say, a larger pixel size, and consequently lower spatial resolution. The opposite is not recommended. If you simply take a signal acquired with a certain GSD, GRD, and in the resampling process "build" a smaller pixel, in which you supposedly want more detailed information, this will not occur (Pixel size <= GSD!!). The signal was already acquired under certain conditions, and you can't change that (GRD) to "detail" the image more. not you can't change that (GRD) to "detail" the image more. In summary, as I've often had to understand in various ways, even to explain to colleagues: If we resample an image to a smaller pixel size, we are NOT improving spatial resolution! If we resample an image to a smaller pixel size, we are NOT improving spatial resolution! If we r esample an image to a smaller pixel size, we are NOT improving spatial resolution ! esample an image to a smaller pixel size, we are NOT improving spatial resolution Another fact is that as users, we end up learning more about pixel size and spatial resolution of an image. It's not common for us to have access to (or seek out) information like GRD or GSD. There's much more to this, but this is a start to clarify the relationship between these concepts, and why pixel size does not necessarily correspond to the spatial resolution of an image! pixel size does not necessarily correspond to the spatial resolution References: *reference 2 was an important reference for this text, for those who want to delve deeper. [1] Spatial Resolution, Pixel Size, and Scale Spatial Resolution, Pixel Size, and Scale [2] The most misunderstood words in Earth Observation The most misunderstood words in Earth Observation [3] Remote Sensing INFORMATION SHEET Remote Sensing INFORMATION SHEET

Pixels and Spatial Resolution in Satellite Images: What's the Real Deal?

I'm puzzled. Product development teams are frustrated by their ways of working. They have a dedicated time slot to discuss and improve things. Yet, it doesn't work. Most teams believe nothing will ever change. As part of product discovery for Codigy Retro, I interviewed 62 companies. I broke down my findings into categories, to see if it would be possible to to solve this. Systematically. Bad experience on the individual/team level: Broken trust. The team raises an issue. It gets lost or left unresolved. No one likes to work without results. If there is no feedback, people will rightly think that it is a pointless exercise.
Peer pressure. When feedback is shared in public, everyone pats each other on the shoulder. It's hard to be the grumpy one who points out something bad.
Poor memory. What did you have for breakfast 4 days ago? We ask people for feedback on the spot, after a hard working week. It's not that we don't have problems to discuss. It's just hard to recall them on the spot.
Lack of structure. Instead of fixing the trust, some facilitators replace problem-solving meeting flow with games and vague discussions. Everyone is engaged, but are there any outcomes? Interactions with Leadership level: Participants raise issues outside the team's scope. That's expected since their quality of life mostly depends on company policies and culture. Topic samples: "Shifting scope", "Lack of product vision", "HR Processes", "Access Policy", "Collaboration with other teams". If we don't address them, we return to the "Broken trust". Why is the high-level stuff usually left unresolved? Your suggestion is discarded as "individual opinion". The team champion gathers insights. Uses all the relationships within the company to share findings. Result? Frequently discarded as an "insignificant issue" or "your opinion”.
"Our tiny team can't change this". Individuals and teams avoid raising issues, thinking it only affects them and is not important enough to be escalated higher. The company is made up of tiny teams. Maybe this issue affects everyone - but we will never find out, because it's never discussed outside the team silo.
Fear of retaliation. Some relationships are not as rosy as we want them to be. Anonymity matters. Especially when rallying support for a controversial topic.
Dev talk vs Business talk. There is plenty of feedback going around in closed peer groups. Devs talk to devs. Leadership talks to leadership. Without a common language and effective communication channels, a meeting between two groups turns into a blaming session. Teams with high autonomy, seem to be happier and not affected by this as much. Maybe that's the long-term solution. How to get out of this situation? After discussing this with the Agile community on Reddit, it seems like teams in this situation have a few options: Continue the struggle. Leave things as they are. Not a happy choice.
Focus team attention on improving the 10% of experience in their control. Barely better. Like cleaning your desk when someone is chopping your arm off. Hard to focus on the small stuff, when your experience is harmed by something bigger.
Go and solve this individually. Build relationships in your organization. Back your argument with team data (anonymized). Try to convert your case into business language. This is by far the best choice. But unfortunately, this is frequently disregarded as "your opinion" because you came alone.
Wait for consultants or leadership to make structural improvements. This may improve things, but when? Take team experience into your own hands The key is to rally support and show that even a small issue can be significant. A different story if it bothers 15 teams or 50 developers. → Gather issues that bother the majority of the teams. Talk with other teams. Extract data summary from retrospectives with Ai, and use data analytics (E.g. Team Health Check trends). → Open a channel/inbox visible to leadership. Submit issues and get public upvotes from other teams. → Convert findings into business language. What are we losing? What could we get, and how? This kind of work would be done by consultants. The benefit of doing it internally, is you get incremental changes NOW. And feels good😎 I'm puzzled. Product development teams are frustrated by their ways of working. They have a dedicated time slot to discuss and improve things. Yet, it doesn't work. Most teams believe nothing will ever change. As part of product discovery for Codigy Retro, I interviewed 62 companies. I broke down my findings into categories, to see if it would be possible to to solve this. Systematically. Bad experience on the individual/team level: Bad experience on the individual/team level: Broken trust. The team raises an issue. It gets lost or left unresolved. No one likes to work without results. If there is no feedback, people will rightly think that it is a pointless exercise. Peer pressure. When feedback is shared in public, everyone pats each other on the shoulder. It's hard to be the grumpy one who points out something bad. Poor memory. What did you have for breakfast 4 days ago? We ask people for feedback on the spot, after a hard working week. It's not that we don't have problems to discuss. It's just hard to recall them on the spot. Lack of structure. Instead of fixing the trust, some facilitators replace problem-solving meeting flow with games and vague discussions. Everyone is engaged, but are there any outcomes? Broken trust. The team raises an issue. It gets lost or left unresolved. No one likes to work without results. If there is no feedback, people will rightly think that it is a pointless exercise. Broken trust. Peer pressure. When feedback is shared in public, everyone pats each other on the shoulder. It's hard to be the grumpy one who points out something bad. Peer pressure. Poor memory. What did you have for breakfast 4 days ago? We ask people for feedback on the spot, after a hard working week. It's not that we don't have problems to discuss. It's just hard to recall them on the spot. Poor memory. Lack of structure . Instead of fixing the trust, some facilitators replace problem-solving meeting flow with games and vague discussions. Everyone is engaged, but are there any outcomes? Lack of structure Interactions with Leadership level: Interactions with Leadership level: Participants raise issues outside the team's scope. That's expected since their quality of life mostly depends on company policies and culture. Topic samples: "Shifting scope", "Lack of product vision", "HR Processes", "Access Policy", "Collaboration with other teams". If we don't address them, we return to the "Broken trust". Why is the high-level stuff usually left unresolved? Your suggestion is discarded as "individual opinion". The team champion gathers insights. Uses all the relationships within the company to share findings. Result? Frequently discarded as an "insignificant issue" or "your opinion”. "Our tiny team can't change this". Individuals and teams avoid raising issues, thinking it only affects them and is not important enough to be escalated higher. The company is made up of tiny teams. Maybe this issue affects everyone - but we will never find out, because it's never discussed outside the team silo. Fear of retaliation. Some relationships are not as rosy as we want them to be. Anonymity matters. Especially when rallying support for a controversial topic. Dev talk vs Business talk. There is plenty of feedback going around in closed peer groups. Devs talk to devs. Leadership talks to leadership. Without a common language and effective communication channels, a meeting between two groups turns into a blaming session. Your suggestion is discarded as "individual opinion". The team champion gathers insights. Uses all the relationships within the company to share findings. Result? Frequently discarded as an "insignificant issue" or "your opinion”. Your suggestion is discarded as "individual opinion". "Our tiny team can't change this". Individuals and teams avoid raising issues, thinking it only affects them and is not important enough to be escalated higher. The company is made up of tiny teams. Maybe this issue affects everyone - but we will never find out, because it's never discussed outside the team silo. "Our tiny team can't change this". Fear of retaliation. Some relationships are not as rosy as we want them to be. Anonymity matters. Especially when rallying support for a controversial topic. Fear of retaliation. Dev talk vs Business talk. There is plenty of feedback going around in closed peer groups. Devs talk to devs. Leadership talks to leadership. Without a common language and effective communication channels, a meeting between two groups turns into a blaming session. Dev talk vs Business talk. Teams with high autonomy, seem to be happier and not affected by this as much. Maybe that's the long-term solution. How to get out of this situation? How to get out of this situation? After discussing this with the Agile community on Reddit , it seems like teams in this situation have a few options: Agile community on Reddit Reddit Continue the struggle. Leave things as they are. Not a happy choice. Focus team attention on improving the 10% of experience in their control. Barely better. Like cleaning your desk when someone is chopping your arm off. Hard to focus on the small stuff, when your experience is harmed by something bigger. Go and solve this individually. Build relationships in your organization. Back your argument with team data (anonymized). Try to convert your case into business language. This is by far the best choice. But unfortunately, this is frequently disregarded as "your opinion" because you came alone. Wait for consultants or leadership to make structural improvements. This may improve things, but when? Continue the struggle. Leave things as they are. Not a happy choice. Continue the struggle. Focus team attention on improving the 10% of experience in their control. Barely better. Like cleaning your desk when someone is chopping your arm off. Hard to focus on the small stuff, when your experience is harmed by something bigger. Focus team attention on improving the 10% of experience in their control. Go and solve this individually. Build relationships in your organization. Back your argument with team data (anonymized). Try to convert your case into business language. This is by far the best choice. But unfortunately, this is frequently disregarded as "your opinion" because you came alone. Go and solve this individually. Wait for consultants or leadership to make structural improvements. This may improve things, but when? Wait for consultants or leadership to make structural improvements. Take team experience into your own hands Take team experience into your own hands The key is to rally support and show that even a small issue can be significant. A different story if it bothers 15 teams or 50 developers. → Gather issues that bother the majority of the teams. Talk with other teams. Extract data summary from retrospectives with Ai, and use data analytics (E.g. Team Health Check trends). Gather issues that bother the majority of the teams. → Open a channel/inbox visible to leadership. Submit issues and get public upvotes from other teams. Open a channel/inbox visible to leadership. → Convert findings into business language. What are we losing? What could we get, and how? Convert findings into business language. This kind of work would be done by consultants. The benefit of doing it internally, is you get incremental changes NOW. And feels good😎

My Teams Hates Retros. What Should I Do?

The main goal of creating decentralization-oriented systems, like cryptocurrencies, was never to make people rich overnight. Instead, they’re aiming to make people free immediately. In these systems, nobody, not even a government or a company, should be able to manipulate your data or your funds by censoring, blocking, or manipulating your transactions. It’s not always like that with blockchains, though. Sadly, most blockchains still have a potential degree of censorship and manipulation due to their own internal workings. There are several steps and middlemen between transaction sending and transaction approval, which open the doors for others to mess around before the final result. Common systems like Proof-of-Work (PoW), used on Bitcoin, or Proof-of-Stake (PoS), used on Ethereum, integrate middlemen known as miners or “validators” to create new coins and approve transactions. Those middlemen have certain power (especially if they collude) to block or manipulate transactions, even in so-called decentralized systems. The reasons to do it could be mostly for profits or to avoid regulatory issues. Let’s see a bit of how they do it. Some types of censorship Blockchains (and similar systems) have several layers or levels in which different things are happening. Think of it like layers in a cake, with each layer doing something different. The network layer connects users, the consensus layer ensures everyone agrees on the state of the blockchain, and the application layer runs programs like smart contracts. Each layer works together to make the whole ecosystem function smoothly. And each layer can have its own type of censorship. Network Layer At the network layer, censorship might happen by restricting who can join or communicate within the peer-to-peer (P2P) network. Two common methods for joining a network —DNS seeding and IP hard-coding— can be censored. DNS seeding involves using domain names to find network participants, while IP hard-coding relies on fixed Internet addresses. If someone blocks access to these domains or IP addresses, they can prevent new users from connecting to the network, effectively censoring who can participate. This kind of censorship is easy to circumvent by using VPN, TOR for access and other seeding methods for peer discovery. So, censorship at the network layer is not very dangerous. Consensus Layer Moving on to the consensus layer, where the main job is to ensure all “validators” or miners agree on the blockchain’s state, censorship might involve the “validators” or miners themselves. For instance, if they decide not to include certain transactions in new blocks or if they prioritize some transactions over others based on who sent them or their content, this is censorship. They have the power to selectively ignore transactions and can be motivated to do so either by internal incentives in the network (making more money), or external pressures, like government’s regulation. This kind of censorship can be circumvented under two conditions: If there are still non-censoring miners/”validators” who would still include the transaction rejected by others,


If the censoring ones are either a minority or apply censorship to their own blocks only and don’t refuse to build on top of blocks that already include the transactions they don’t like. However, if the censoring miners or ”validators” both have the majority and refuse to build on top of blocks that happen to include the transactions they don’t like, such transactions would be completely blocked from the blockchain. Thus, censorship at the consensus layer is very dangerous for digital freedoms. Application Layer + Additional Services Lastly, at the application layer, censorship can occur through some decentralized applications (Dapps) that are not really decentralized. These Dapps, like smart contracts, have rules for how they operate. If these rules allow certain entities (companies, DAOs) to restrict certain users or types of transactions, or even to change the rules themselves (upgradeable contracts), this can be another form of censorship. For example, the operators of the biggest stablecoins, USDT and USDC, are able to freeze user accounts, and they have done so multiple times. Additionally, external services like wallets and user interfaces (e.g., websites) could also impose restrictions, affecting how users interact with the chain. Censorship at the Dapp layer can’t be circumvented at all. Censorship at the wallet and user interface level is generally not dangerous as it can be circumvented by moving one’s private keys to another wallet or using alternative user interfaces. DAG Evolution Obyte, as a Directed Acyclic Graph (DAG) system, offers a unique approach that makes it more resistant to censorship than blockchains. In Obyte, transactions are added directly by users without needing approval from miners, “validators”, or any centralized authority. This means that no single entity or group of entities has the power to block or alter transactions, allowing for a more decentralized and open system. Since each transaction automatically becomes a part of the DAG, it’s secure and unchangeable, ensuring that all users can freely participate without fear of interference. Moreover, while blockchains rely on “validators” or miners to order and confirm transactions, which can sometimes lead to censorship, Obyte uses a different method. Order Providers (OPs) post guiding transactions to help establish a consensus order, but they don’t have the power to change the history or reject transactions. This setup removes middlemen and eliminates the risk of censorship, making sure user funds can never be confiscated or frozen and users are always free to use them however they like. When it comes to external services like wallets or oracles, Obyte’s decentralized nature encourages the development of multiple, independent, and open-source services. Users aren’t tied to any specific service provider, reducing the risk of censorship. Overall, Obyte’s DAG structure empowers users by giving them full control over their transactions, providing a much more censorship-resistant and autonomous experience compared to blockchain networks. Featured Vector Image by pikisuperstar / Freepik The main goal of creating decentralization-oriented systems, like cryptocurrencies, was never to make people rich overnight. Instead, they’re aiming to make people free immediately. In these systems, nobody, not even a government or a company, should be able to manipulate your data or your funds by censoring, blocking, or manipulating your transactions. It’s not always like that with blockchains, though. Sadly, most blockchains still have a potential degree of censorship and manipulation due to their own internal workings. There are several steps and middlemen between transaction sending and transaction approval, which open the doors for others to mess around before the final result. Common systems like Proof-of-Work (PoW), used on Bitcoin, or Proof-of-Stake (PoS), used on Ethereum, integrate middlemen known as miners or “validators” to create new coins and approve transactions. Sadly, most blockchains still have a potential degree of censorship and manipulation due to their own internal workings. Those middlemen have certain power (especially if they collude) to block or manipulate transactions, even in so-called decentralized systems. The reasons to do it could be mostly for profits or to avoid regulatory issues. Let’s see a bit of how they do it. Some types of censorship Blockchains (and similar systems) have several layers or levels in which different things are happening. Think of it like layers in a cake, with each layer doing something different. The network layer connects users, the consensus layer ensures everyone agrees on the state of the blockchain, and the application layer runs programs like smart contracts. Each layer works together to make the whole ecosystem function smoothly. And each layer can have its own type of censorship . its own type of censorship its own type of censorship Network Layer At the network layer, censorship might happen by restricting who can join or communicate within the peer-to-peer (P2P) network. Two common methods for joining a network —DNS seeding and IP hard-coding— can be censored. DNS seeding involves using domain names to find network participants, while IP hard-coding relies on fixed Internet addresses. If someone blocks access to these domains or IP addresses, they can prevent new users from connecting to the network, effectively censoring who can participate. At the network layer, censorship might happen by restricting who can join or communicate within the peer-to-peer (P2P) network. This kind of censorship is easy to circumvent by using VPN, TOR for access and other seeding methods for peer discovery. So, censorship at the network layer is not very dangerous. Consensus Layer Moving on to the consensus layer, where the main job is to ensure all “validators” or miners agree on the blockchain’s state, censorship might involve the “validators” or miners themselves . For instance, if they decide not to include certain transactions in new blocks or if they prioritize some transactions over others based on who sent them or their content, this is censorship. They have the power to selectively ignore transactions and can be motivated to do so either by internal incentives in the network (making more money), or external pressures, like government’s regulation. Moving on to the consensus layer, where the main job is to ensure all “validators” or miners agree on the blockchain’s state, censorship might involve the “validators” or miners themselves This kind of censorship can be circumvented under two conditions: If there are still non-censoring miners/”validators” who would still include the transaction rejected by others, If the censoring ones are either a minority or apply censorship to their own blocks only and don’t refuse to build on top of blocks that already include the transactions they don’t like. If there are still non-censoring miners/”validators” who would still include the transaction rejected by others, If there are still non-censoring miners/”validators” who would still include the transaction rejected by others, If the censoring ones are either a minority or apply censorship to their own blocks only and don’t refuse to build on top of blocks that already include the transactions they don’t like. If the censoring ones are either a minority or apply censorship to their own blocks only and don’t refuse to build on top of blocks that already include the transactions they don’t like. However, if the censoring miners or ”validators” both have the majority and refuse to build on top of blocks that happen to include the transactions they don’t like, such transactions would be completely blocked from the blockchain. Thus, censorship at the consensus layer is very dangerous for digital freedoms. Application Layer + Additional Services Lastly, at the application layer, censorship can occur through some decentralized applications (Dapps) that are not really decentralized. These Dapps, like smart contracts, have rules for how they operate. If these rules allow certain entities (companies, DAOs) to restrict certain users or types of transactions, or even to change the rules themselves (upgradeable contracts), this can be another form of censorship. For example, the operators of the biggest stablecoins, USDT and USDC, are able to freeze user accounts, and they have done so multiple times. Lastly, at the application layer, censorship can occur through some decentralized applications (Dapps) that are not really decentralized. they have done so they have done so Additionally, external services like wallets and user interfaces (e.g., websites) could also impose restrictions, affecting how users interact with the chain. Censorship at the Dapp layer can’t be circumvented at all. Censorship at the wallet and user interface level is generally not dangerous as it can be circumvented by moving one’s private keys to another wallet or using alternative user interfaces. DAG Evolution Obyte , as a Directed Acyclic Graph (DAG) system, offers a unique approach that makes it more resistant to censorship than blockchains. In Obyte, transactions are added directly by users without needing approval from miners, “validators”, or any centralized authority. This means that no single entity or group of entities has the power to block or alter transactions, allowing for a more decentralized and open system. Since each transaction automatically becomes a part of the DAG, it’s secure and unchangeable, ensuring that all users can freely participate without fear of interference. Obyte Obyte In Obyte, transactions are added directly by users without needing approval from miners, “validators”, or any centralized authority. directly by users Moreover, while blockchains rely on “validators” or miners to order and confirm transactions, which can sometimes lead to censorship, Obyte uses a different method. Order Providers (OPs) post guiding transactions to help establish a consensus order, but they don’t have the power to change the history or reject transactions. This setup removes middlemen and eliminates the risk of censorship, making sure user funds can never be confiscated or frozen and users are always free to use them however they like. Order Providers Order Providers When it comes to external services like wallets or oracles, Obyte’s decentralized nature encourages the development of multiple, independent, and open-source services. Users aren’t tied to any specific service provider, reducing the risk of censorship. Overall, Obyte’s DAG structure empowers users by giving them full control over their transactions, providing a much more censorship-resistant and autonomous experience compared to blockchain networks. Featured Vector Image by pikisuperstar / Freepik Featured Vector Image by pikisuperstar / Freepik Freepik Freepik

Educational Byte: Why Blockchains Can Be Censored and Manipulated?

Opvoedkundige byte: waarom blokkettings gesensor en gemanipuleer kan word?

የትምህርት ባይት፡ ለምን ብሎክቼይን ሳንሱር ሊደረግ እና ሊታለል ይችላል?

Byte educativo: ¿Por qué las cadenas de bloques pueden ser censuradas y manipuladas?

Edikasyon Byte: Poukisa Blockchains ka Sansi ak Manipile?

教育バイト: ブロックチェーンが検閲され、操作される可能性があるのはなぜですか?

Mokomasis baitas: kodėl blokų grandinės gali būti cenzūruojamos ir manipuliuojamos?

تعلیمي بایټ: ولې بلاکچینونه سانسور او مینځل کیدی شي؟

Byte Educativo: ¿Imaraykutaq Blockchains nisqakunata censurayta hinaspa manipulayta atikunman?

Vzdelávací bajt: Prečo môžu byť blockchainy cenzurované a manipulované?

Образовни бајт: Зашто се блокчејн може цензурисати и манипулисати?

Навчальний байт: чому блокчейни можна цензурувати та маніпулювати ними?

教育字节：为什么区块链会被审查和操纵？

Architecting Scalable Databases for Large-Scale Systems

Authors:
(1) Yuxuan Yan,Tencent with Equal contributions and yuxuanyan@tencent.com;
(2) Chi Zhang, Tencent with Equal contributions and Corresponding Author, johnczhang@tencent.com;
(3) Rui Wang, Tencent and raywwang@tencent.com;
(4) Yichao Zhou, Tencent and yichaozhou@tencent.com;
(5) Gege Zhang, Tencent and gretazhang@tencent.com;
(6) Pei Cheng, Tencent and peicheng@tencent.com;
(7) Bin Fu, Tencent and brianfu@tencent.com;
(8) Gang Yu, Tencentm and skicyyu@tencent.com. Table of Links Abstract and 1 Introduction 2. Related Work 3. Method and 3.1. Hybrid Guidance Strategy 3.2. Handling Multiple Identities 3.3. Training 4. Experiments 4.1. Implementation details. 4.2. Results 5. Conclusion and References 2. Related Work Text-to-image diffusion models. Diffusion models have recently come to the forefront of generative model research. Their exceptional capability to generate high-quality and diverse images has placed them at a distinct advantage over their predecessors, namely the GAN-based and autoregressive image generation models. This new generation of diffusion models, having the capacity to produce stateof-the-art synthesis results, owes much of its success to being trained on image-text pairs data at a billion scale. The integration of textual prompts into the diffusion model serves as a vital ingredient in the development of various text-to-image diffusion models. Distinguished models in this domain include GLIDE [38], DALL·E 2 [41], Imagen [45], and StableDiffusion [42]. These models leverage text as guidance during the image generation process. Consequently, they have shown considerable proficiency in synthesizing high-quality images that closely align with the provided textual description. Compared to previous efforts [29, 39, 51, 69] on face image synthesis based on Generative Adversarial Networks (GANs) [25], diffusion models exhibit greater stability during training and demonstrate enhanced capability in effectively integrating diverse forms of guidance, such as texts and stylized images. However, textual descriptions for guidance, while immensely useful, often fall short when it comes to the generation of complex and nuanced details that are frequently associated with certain subjects. For instance, generating images of human faces proves challenging with this approach. While generating images of celebrities might be feasible due to the substantial presence of their photographs in the training data that can be linked to their names, it becomes an uphill task when it comes to generating images of ordinary people using these text-to-diffusion models. Subject-driven image generation. Subject-driven image generation seeks to overcome the limitations posed by text-to-image synthesis models. Central to this novel research area is the inclusion of subject-specific reference images, which supplement the textual description to yield more precise and personalized image synthesis. To this end, several optimization-based methods have been employed, with popular ones being Low-Rank Adaptation (LoRA), DreamBooth [43], Textual Inversion [14], and Hypernetwork [3]. These methods typically involve fine-tuning a pre-trained text-to-image framework or textual embeddings to align the existing model with user-defined concepts, as indicated by a set of example images. There are several other methods derived from these works such as Unitune [49], HyperDreamBooth [44], EasyPhoto [53], FaceChain [33], etc. [4, 23, 60]. However, these methods pose some challenges. For one, they are time-consuming due to the need for model fine-tuning, which also requires multiple reference images to achieve accurate fitting. Overfitting also becomes a potential issue with these optimization-based methods, given their heavy reliance on example images. In response to these challenges, recent studies have proposed various improved methods. DreamArtist [13], for instance, mitigates the problem of overfitting by incorporating both positive and negative embeddings and jointly training them. Similarly, E4T [15] introduced an encoder-based domain-tuning method to accelerate the personalization of text-to-image models, offering a faster solution to model fitting for new subjects. There are large numbers of similar methods which are encoder-based including Composer [21], ProFusion [68] MagiCapture [22], IP-Adapter [59], ELITE [52], DisenBooth [8], Face0 [50], PhotoVerse [9], AnyDoor [11], SingleInsert [54], etc. [24, 35, 56]. Alongside optimization-based methods, a number of tuning-free methods have been concurrently proposed, such as InstantBooth [47]. This method converts input images into a textual token for general concept learning and introduces adapter layers to obtain rich image representation for generating fine-grained identity details. However, it comes with the drawback of having to train the model separately for each category. Bansal et al.[6] put forward a universal guidance algorithm that enables diffusion models to be controlled by arbitrary guidance modalities without the need to retrain the network. Similarly, Yang et al.[57] propose an inpainting-like diffusion model that learns to inpaint a masked image under the guidance of example images of a given subject. Similar methods leveraged by inversion-based personalization including Cao et al. [7], Han et al. [17], Gu et al. [16], Mokady et al. [37]. Besides, Controlnet [67] is also an effective way to personalize. In addition, there are some solutions that use editing to achieve the purpose of maintaining identity, such as SDEdit [36], Imagic [26], etc. [18, 48]. Based on all the solutions mentioned above, the method of combining multiple objects while maintaining identity is also widely used in Break-a-scene [5], FastComposer [55], Cones [34] and MultiConceptCustomDiffusion [27]. Within the text-to-image community, one specific area of interest that has garnered significant attention is the generation of human images. Human-centric image generation, owing to its vast applicability and widespread popularity, constitutes a substantial proportion of posts in the community, such as Civitai [1]. In light of these considerations, our work primarily focuses on the preservation of human identity during the image generation process. Few-shot Learning. It is a common practice in the field of few-shot learning literature to convert optimizationbased tasks into a feed-forward approach, which can significantly reduce computation time. Numerous previous studies in the realm of few-shot learning have adopted this approach to address computer vision tasks, including but not limited to classification [63–66], detection [58], and segmentation [10, 28, 30–32, 61, 62]. In this research, our objective aligns with this approach as we aim to integrate identity information into the generative process through a feed-forward module. This paper is available on arxiv under CC0 1.0 DEED license. Authors: (1) Yuxuan Yan,Tencent with Equal contributions and yuxuanyan@tencent.com; (2) Chi Zhang, Tencent with Equal contributions and Corresponding Author, johnczhang@tencent.com; (3) Rui Wang, Tencent and raywwang@tencent.com; (4) Yichao Zhou, Tencent and yichaozhou@tencent.com; (5) Gege Zhang, Tencent and gretazhang@tencent.com; (6) Pei Cheng, Tencent and peicheng@tencent.com; (7) Bin Fu, Tencent and brianfu@tencent.com; (8) Gang Yu, Tencentm and skicyyu@tencent.com. Authors: Authors: (1) Yuxuan Yan,Tencent with Equal contributions and yuxuanyan@tencent.com; (2) Chi Zhang, Tencent with Equal contributions and Corresponding Author, johnczhang@tencent.com; (3) Rui Wang, Tencent and raywwang@tencent.com; (4) Yichao Zhou, Tencent and yichaozhou@tencent.com; (5) Gege Zhang, Tencent and gretazhang@tencent.com; (6) Pei Cheng, Tencent and peicheng@tencent.com; (7) Bin Fu, Tencent and brianfu@tencent.com; (8) Gang Yu, Tencentm and skicyyu@tencent.com. Table of Links Abstract and 1 Introduction Abstract and 1 Introduction 2. Related Work 2. Related Work 3. Method and 3.1. Hybrid Guidance Strategy 3. Method and 3.1. Hybrid Guidance Strategy 3.2. Handling Multiple Identities 3.2. Handling Multiple Identities 3.3. Training 3.3. Training 4. Experiments 4.1. Implementation details. 4.1. Implementation details. 4.2. Results 4.2. Results 5. Conclusion and References 5. Conclusion and References 2. Related Work Text-to-image diffusion models. Diffusion models have recently come to the forefront of generative model research. Their exceptional capability to generate high-quality and diverse images has placed them at a distinct advantage over their predecessors, namely the GAN-based and autoregressive image generation models. This new generation of diffusion models, having the capacity to produce stateof-the-art synthesis results, owes much of its success to being trained on image-text pairs data at a billion scale. The integration of textual prompts into the diffusion model serves as a vital ingredient in the development of various text-to-image diffusion models. Distinguished models in this domain include GLIDE [38], DALL·E 2 [41], Imagen [45], and StableDiffusion [42]. These models leverage text as guidance during the image generation process. Consequently, they have shown considerable proficiency in synthesizing high-quality images that closely align with the provided textual description. Compared to previous efforts [29, 39, 51, 69] on face image synthesis based on Generative Adversarial Networks (GANs) [25], diffusion models exhibit greater stability during training and demonstrate enhanced capability in effectively integrating diverse forms of guidance, such as texts and stylized images. Text-to-image diffusion models. However, textual descriptions for guidance, while immensely useful, often fall short when it comes to the generation of complex and nuanced details that are frequently associated with certain subjects. For instance, generating images of human faces proves challenging with this approach. While generating images of celebrities might be feasible due to the substantial presence of their photographs in the training data that can be linked to their names, it becomes an uphill task when it comes to generating images of ordinary people using these text-to-diffusion models. Subject-driven image generation . Subject-driven image generation seeks to overcome the limitations posed by text-to-image synthesis models. Central to this novel research area is the inclusion of subject-specific reference images, which supplement the textual description to yield more precise and personalized image synthesis. To this end, several optimization-based methods have been employed, with popular ones being Low-Rank Adaptation (LoRA), DreamBooth [43], Textual Inversion [14], and Hypernetwork [3]. These methods typically involve fine-tuning a pre-trained text-to-image framework or textual embeddings to align the existing model with user-defined concepts, as indicated by a set of example images. There are several other methods derived from these works such as Unitune [49], HyperDreamBooth [44], EasyPhoto [53], FaceChain [33], etc. [4, 23, 60]. However, these methods pose some challenges. For one, they are time-consuming due to the need for model fine-tuning, which also requires multiple reference images to achieve accurate fitting. Overfitting also becomes a potential issue with these optimization-based methods, given their heavy reliance on example images. In response to these challenges, recent studies have proposed various improved methods. DreamArtist [13], for instance, mitigates the problem of overfitting by incorporating both positive and negative embeddings and jointly training them. Similarly, E4T [15] introduced an encoder-based domain-tuning method to accelerate the personalization of text-to-image models, offering a faster solution to model fitting for new subjects. There are large numbers of similar methods which are encoder-based including Composer [21], ProFusion [68] MagiCapture [22], IP-Adapter [59], ELITE [52], DisenBooth [8], Face0 [50], PhotoVerse [9], AnyDoor [11], SingleInsert [54], etc. [24, 35, 56]. Subject-driven image generation Alongside optimization-based methods, a number of tuning-free methods have been concurrently proposed, such as InstantBooth [47]. This method converts input images into a textual token for general concept learning and introduces adapter layers to obtain rich image representation for generating fine-grained identity details. However, it comes with the drawback of having to train the model separately for each category. Bansal et al.[6] put forward a universal guidance algorithm that enables diffusion models to be controlled by arbitrary guidance modalities without the need to retrain the network. Similarly, Yang et al.[57] propose an inpainting-like diffusion model that learns to inpaint a masked image under the guidance of example images of a given subject. Similar methods leveraged by inversion-based personalization including Cao et al. [7], Han et al. [17], Gu et al. [16], Mokady et al. [37]. Besides, Controlnet [67] is also an effective way to personalize. In addition, there are some solutions that use editing to achieve the purpose of maintaining identity, such as SDEdit [36], Imagic [26], etc. [18, 48]. Based on all the solutions mentioned above, the method of combining multiple objects while maintaining identity is also widely used in Break-a-scene [5], FastComposer [55], Cones [34] and MultiConceptCustomDiffusion [27]. Within the text-to-image community, one specific area of interest that has garnered significant attention is the generation of human images. Human-centric image generation, owing to its vast applicability and widespread popularity, constitutes a substantial proportion of posts in the community, such as Civitai [1]. In light of these considerations, our work primarily focuses on the preservation of human identity during the image generation process. Few-shot Learning. It is a common practice in the field of few-shot learning literature to convert optimizationbased tasks into a feed-forward approach, which can significantly reduce computation time. Numerous previous studies in the realm of few-shot learning have adopted this approach to address computer vision tasks, including but not limited to classification [63–66], detection [58], and segmentation [10, 28, 30–32, 61, 62]. In this research, our objective aligns with this approach as we aim to integrate identity information into the generative process through a feed-forward module. Few-shot Learning. This paper is available on arxiv under CC0 1.0 DEED license. This paper is available on arxiv under CC0 1.0 DEED license. available on arxiv

FaceStudio: Put Your Face Everywhere in Seconds: Related Work

So You Want To Be On A Startup Board of Directors

Why You Should Forget Everything You Know About Problem Solving

The ideology serves as a powerful mechanism not only for uniting individuals with similar interests but also for exchanging experiences and ideas, embodying the principle that sharing is indeed caring. is a great example of how a community of enthusiasts can change the world. Its are highly impressive and we can claim that most of the Internet works on Linux’s horsepower. open source Linux usage statistics Bad programmers worry about the code. Good programmers worry about data structures and their relationships. Linus Torvalds Managing an open-source project carries significant weight, particularly as its popularity grows: it brings forth the spotlight on technical expertise and architectural choices. Popular projects could be a treasure trove of knowledge for anyone who wants to boost their skills. Let's delve deeply into a few notable projects. Why focus on open source? First of all, it embodies top-tier coding standards, integrating best practices championed by industry leaders. My firm belief lies in the notion that genuine programmers are fueled by passion and curiosity. Why not learn something new and integrate it into our daily practice? Secondly, hacking code could take your skills to a whole new level if you are consistent and tenacious. Last but not least, it could help you jump right into the project you like: find new buddies, brand yourself as a skilled developer, and get the job of your dreams. Help the community to solve their problems, and the reward will not keep you waiting. Take your cup of coffee, let’s get straight to work. Linux Kernel The very heart of the Linux system — its kernel. Indeed, this is not merely a singular project, but rather a constellation of diverse projects unified by a central idea: tying up different parts of hardware and providing a sustainable, secure API for the end users. The Linux codebase is huge and contains a lot of vendor-specific code, but it is well-organized and structured. I strongly recommend everyone interested to read . It might be a bit outdated but it will foster a solid understanding of how the kernel internals work. The kernel version I have on the table is 6.3. this Radix tree for memory management At the very beginning of the main.c we can spot definitions of some functions like . It is responsible for initializing the data structure that is used in memory management processes. radix_tree_init In the codebase, the radix_tree_root struct is an alias for a wrapper above the struct. It a lock (pthread mutex); xa_flags — the enum that tells how memory allocations will be performed; and the pointer to xa_head. XArray holds gfp_t struct xarray {
	spinlock_t	xa_lock;
/* private: The rest of the data structure is not to be used directly. */
	gfp_t		xa_flags;
	void __rcu *	xa_head;
}; typedef enum {
	GFP_KERNEL,
	GFP_ATOMIC,
	__GFP_HIGHMEM,
	__GFP_HIGH
} gfp_t; If all of the entries in the array are NULL, xa_head is a NULL pointer. If the only non-NULL entry in the array is at index 0, xa_head is that entry. If any other entry in the array is non-NULL, xa_head points to a xa_node — the internal struct that handles offset, counts, slots, and other fields. struct xa_node {
	unsigned char	shift;		/* Bits remaining in each slot */
	unsigned char	offset;		/* Slot offset in parent */
	unsigned char	count;		/* Total entry count */
	unsigned char	nr_values;	/* Value entry count */
	struct xa_node __rcu *parent;	/* NULL at top of tree */
	struct xarray	*array;		/* The array we belong to */
	union {
		struct list_head private_list;	/* For tree user */
		struct rcu_head	rcu_head;	/* Used when freeing node */
	};
	void __rcu	*slots[XA_CHUNK_SIZE];
	union {
		unsigned long	tags[XA_MAX_MARKS][XA_MARK_LONGS];
		unsigned long	marks[XA_MAX_MARKS][XA_MARK_LONGS];
	};
}; The radix tree data structure contains “nodes” that link with each other based on the int key (index). The node contains several slots, each of which can handle a pointer. The main idea here is to keep a tree wide, not deep — it is effective for fast addressing. When the function is called by a user (it can be a device driver), the following actions occur: radix_tree_insert __radix_create_tree creates a slot in the tree if it is necessary, based on the passed index which is used to check the size of the tree. Or raise an error if there is no available memory; insert_entities pass an item to the slot; some asserts check that offsets are applied correctly to the data structure. Wherever a user of the tree is going to look up a specific page, they need to look up it with an index. The lookup phase is quite efficient because of the broad nature of the structure. It is based on the key that is modified on the user side. Take a look at the for more insight into how it works. article Users of this data structure include GPU and network device drivers, which maintain memory page pointers in a structured format rather than juggling them. It's like having a pocket dimension! This is where Linus's earlier quote comes into play. Consider the essence of the data you handle; aim for efficiency, compactness, and cleanliness in its usage, regardless of the language or project you're involved in. In 2021 Liam Howlett (no, another Liam) and Matthew Wilcox introduced data structure. I encourage you to read more about the intentions, motivation, and realization details and . the maple tree here here Kubernetes Let's direct our steps to the point of the architectural design. One of the most famous orchestrators, invented by Google and spread around the world at rocket pace — Kubernetes. There are a lot of fairly new resources out there covering the internal details, for example here is a that you can start exploring as you delve deeper into the details. repository The whole point of this article is to get some useful insights that answer not just “how” but “why”. Understanding the motivation is crucial, in my opinion, that is how we can “catch a mindset” to re-implement all those cool staff in our daily job. Let’s briefly recall what the Kubernetes is. This is the system that manages containers within the applications and provides orchestration features such as scaling and high availability. Kubernetes’ main components are the API service, the scheduler, etcd — the key-value storage, the controller service, and the agent service — “kubelete” on each node to manage containers. Guys from developed it to utilize the compute resources of their cloud . Their main intention was a “great container management system”. Google as efficiently as possible Architectural highlights The very first thing I’d like to highlight is the distributed nature of execution. To synchronize all of the user’s entities states the orchestrator uses etcd — the key-value storage which information in a tree data structure. It is a common pattern that presents a to manage different states between multiple entities, with one slight difference: the storage is also used to “instruct” all of the components about the final cluster’s state. keeps So the whole system uses the storage as a source of truth to keep maintaining the desired state of a cluster. source of truth One more nice architectural decision is the — container runtime interface. CRI It allows Kubernetes to work not just with Docker but many others (CRI-O, Dockerd) by defining the , just like one from RFC. The kubelet agent utilizes GRPC to connect with a runtime, so in case one would like to implement its own runtime, they should implement the exact interface on their side. The concept aims to loosen the connection between the kubelet agent and a runtime which manages containers. interaction protocol At the end let’s briefly look at the “sidecar” pattern that was presented by third-party services like and has recently been within the core Kubernetes functionality. Istio implemented The whole idea is to embed a service to the main application that helps to handle some specific cases such as proxy or load-balancing. Kubernetes manages this container as a whole different entity of a pod: the container can the entire pod status; restarting and updating separately of the main application; resources with the main app. influence share The main point here is that the main application may know nothing about the sidecar, while the behavior of the entire service can be changed. Bless Open Source, we how new functionality was implemented by the community. can trace PostgreSQL PostgreSQL is not just a database but a whole platform. After the client connects to the server a new is established. The query goes through many while executed: parsing, the rewriting system, planning and executing. backend process stages The PSQL server has a bunch of that allow developers to interact with different internal components. For example, we can our scan provider aimed, for some optimizations at each stage. Or create an . Even though the overall process required for specific knowledge and the documentation could be a bit knotty, interfaces write index add-on the loosely coupled architecture empowers developers to vastly expand functionality. There are many that we can use within the platform and the “bloom index” is one of them. extensions realization Bloom index I faced a problem of growing indexes back in the day. The task was to implement the custom search filter: every column in the database and the combination of it could be a key for a search. Building indexes for each and every column could be a bad decision because of the complexity and size. For example, if we are sure that item==2 is False there is no reason to check all other fields. I remembered the — the probabilistic data structure that could return a false-positive result (because of hash collision) and be quite sure when the data is absent. As I’ve mentioned earlier PSQL has an extension for this data structure. The very first idea that comes to mind is to reduce the dimension of possible variations removing the query values that we are sure are not in the database. Bloom filter Bloom filter in a nutshell — it’s just a bit array. Every item would be hashed with a certain amount of different functions that translates it to some index. That index will set the exact bit in the filter’s array. In the case of 3 different functions, an item would set 3 different bits. At the lookup phase, the query would be also hashed and compared within the bits in the array. If at least one of the bits is not set, we can be sure the item is absent. If all of them were set, it is still possible that the item was not placed — that is because each bit we have found could refer to other items — the collision takes place. Check out this great of how it works within the examples. explanation To start using the bloom index within the PSQL we need to create it properly: CREATE INDEX bloom_idx_bar ON foo.bar USING bloom (id,dept_id,zipcode) WITH (length=80, col1=4, col2=2, col3=4); where “length” — is the length of the array of bits for each row; and the “col” numbers — describe how many bits would be occupied by the column (how many functions would be applied). By increasing these numbers we can reduce false-positive shots. Of index creation in PSQL is a bit complicated, we should understand internal data structures and protocols; let’s try to make the observation short and sweet. The function promotes the data structure’s functions as an IndexAmRoutine pointer — a pointer to the , that represents an API for the index access methods. blhandler struct The function creates the new index: allocate memory pages, scan the table for tuples that will be indexed and in the structure. blbuild BloomState typedef struct BloomState
{
	FmgrInfo	hashFn[INDEX_MAX_KEYS];
	Oid			collations[INDEX_MAX_KEYS];
	BloomOptions opts;			/* copy of options on index's metapage */
	int32		nColumns;

	/*
	 * sizeOfBloomTuple is index-specific, and it depends on reloptions, so
	 * precompute it
	 */
	Size		sizeOfBloomTuple;
} BloomState; When we insert data to the index, we pass it in the form of an array within the array of booleans that marks NULL values to skip it at the indexing phase. The BloomFormTuple function forms the BloomTuple struct from the data — this is the part where the data would be signed. The tuple would be passed to a memory page after that phase. BloomTuple *
BloomFormTuple(BloomState *state, ItemPointer iptr, Datum *values, bool *isnull)
{
	int			i;
	BloomTuple *res = (BloomTuple *) palloc0(state->sizeOfBloomTuple);

	res->heapPtr = *iptr;

	/* Blooming each column */
	for (i = 0; i < state->nColumns; i++)
	{
		/* skip nulls */
		if (isnull[i])
			continue;

		signValue(state, res->sign, values[i], i);
	}

	return res;
} At the scan phase the forms the struct which contains the index’s data and the makes the TIDBitmap struct from it — the internal data structure that holds item pointers — reading BloomState page by page. blbeginscan IndexScanDesc blgetbitmap I hope the explanation isn't too confusing, PSQL internals are quite specific. Bloom filter is a pretty common data structure, PSQL uses the same approach for some purposes. It is also broadly used in many different situations: from cache-hitting cases to implementations. internal kernel packet filter More to know In the article, I’ve just lifted the veil of the shiny open-source world. There are many more great implementations you could meet in the wild: and — if you’d like to catch the magic of the “fair” resource allocation; — to understand the simultaneous editing in Google Docs. CFS CFQ operational transform and CRDT The whole software world is in motion: sidecars and operators to be the new model for software delivery; eBPF is in a to change the service-mesh post. claimed hurry Take a look at the elegance of the great things’ internals.

Algorithms in the Wild: Reusable Insights and Strategies from the Open Source Community

Stories, News & Mentions about NOTCOMMON

Notcommon’s Stories on HackerNoon