Banks in the EU are gearing up for the implementation of new capital requirement rules when Basel 4 takes effect on January 1 next year, but doubts remain over the treatment of key trade finance products. Even though the deadline is just days away, there are still differing interpretations ...

EU trade finance banks puzzle over Basel rules as deadline looms | Global Trade Review (GTR)

Coote has been suspended by refereeing body Professional Game Match Officials Limited (PGMOL) with immediate effect pending a full investigation, while the Football Association has launched its own investigation.

'David Coote video on Liverpool and Jurgen Klopp fuels conspiracy theorists' - BBC Sport

Niels Wittich leaves his role as Formula 1's race director with immediate effect.

Wittich steps down as F1 race director - Yahoo Sports

Judgement raises prospect of expensive compensation scheme.

DCA court ruling could have major effect on finance industry - Motor Trade News

Key compliance dates for New York's updated cybersecurity regulations take effect on November 1, impacting Class A companies, covered entities, and small businesses.

NYDFS Cybersecurity Regulation Compliance Requirements for November 1, 2024

The library posted the news on its website, stating the closures took effect as of 5 p.m. Friday.

Calgary Public Library locations closed due to cybersecurity breach - Calgary | Globalnews.ca

"The Iranian military has responded by saying any attack from Israel would trigger yet a stronger response from Iran, so the Geopolitical stages effect on crude continues to grow," Dennis Kissler, BOK Financial's senior vice president of trading, wrote on Monday.

Stock market today: Dow, S&P 500, Nasdaq slip as investors rethink rate cut 
bets

Five climate–energy–economy models are used to explore the effect of reducing the cost gap in energy financing between developed and developing countries through fair-finance. Such convergence is projected to increase energy availability, affordability, and sustainability in developing ...

Fair energy finance increases global equity in the green energy transition | Nature Energy

effect

Authors:
(1) Hoon Kim, Beeble AI, and contributed equally to this work;
(2) Minje Jang, Beeble AI, and contributed equally to this work;
(3) Wonjun Yoon, Beeble AI, and contributed equally to this work;
(4) Jisoo Lee, Beeble AI, and contributed equally to this work;
(5) Donghyun Na, Beeble AI, and contributed equally to this work;
(6) Sanghyun Woo, New York University, and contributed equally to this work. Editor's Note: This is Part 2 of 14 of a study introducing a method for improving how light and shadows can be applied to human portraits in digital images. Read the rest below. Table of Links Abstract and 1. Introduction
2. Related Work
3. SwitchLight and 3.1. Preliminaries
3.2. Problem Formulation
3.3. Architecture
3.4. Objectives
4. Multi-Masked Autoencoder Pre-training
5. Data
6. Experiments
7. Conclusion Appendix A. Implementation Details
B. User Study Interface
C. Video Demonstration
D. Additional Qualitative Results & References 2. Related Work Human Portrait Relighting is an ill-posed problem due to its under-constrained nature. To tackle this, earlier methods incorporated 3D facial priors [44], exploited image intrinsics [3, 40], or framed the task as a style transfer [43]. Light stage techniques [49] offer a more powerful solution by recording subject’s reflectance fields under varying lighting conditions [10, 14], though they are labor-intensive and require specialized equipment. A promising alternative has emerged with deep learning, utilizing neural networks trained on light stage data. Sun et al. [45] pioneered this approach, but their method had limitations in representing non-Lambertian effects. This was improved upon by Nestmeyer et al. [32], who integrated rendering physics into network design, albeit limited to directional light. Building upon this, Pandey et al. [34] incorporated the Phong reflection model and a high dynamic range (HDR) lighting map [9] into their network, enabling a more accurate representation of global illumination. Simultaneously, efforts have been made to explore portrait relighting without light stage data [20, 21, 42, 47, 55]. Moreover, introduction of NeRF [7] and diffusion-based [38] models has opened new avenues in the field. However, networks trained with lightstage data maintain superior accuracy and realism, thanks to physics-based composited relight image training pairs and precise ground truth image intrinsics [56]. Our work furthers this domain by integrating the Cook-Torrance model into our network design, shifting from the empirical Phong model to a more physics-based approach, thereby enhancing the realism and detail in relit images. Self-supervised Pre-training has become a standard training scheme in the development of large language models like BERT [11] and GPT [39], and is increasingly influential in vision models, aiming to replicate the ‘BERT moment’. This approach typically involves pre-training on extensive unlabeled data, followed by fine-tuning on specific tasks. While early efforts in vision models focused on simple pretext tasks [13, 17, 33, 36, 53], the field has evolved through stages like contrastive learning [5, 18] and masked image modeling [2, 19, 50]. However, the primary focus has remained on visual recognition, with less attention to other domains. Exceptions include low-level image processing tasks [4, 6, 27, 30] using the vision transformer [15]. Our research takes a different route, focusing on human portrait relighting—a complex challenge of manipulating illumination in the image. This direction is crucial because acquiring accurate ground truth data, especially from light stage, is both expensive and difficult. We modify the MAE framework [19], previously successful in robust image representation learning and developing locality biases [35], to suit the unique requirements of effective relighting. This paper is available on arxiv under CC BY-NC-SA 4.0 DEED license. Authors: (1) Hoon Kim, Beeble AI, and contributed equally to this work; (2) Minje Jang, Beeble AI, and contributed equally to this work; (3) Wonjun Yoon, Beeble AI, and contributed equally to this work; (4) Jisoo Lee, Beeble AI, and contributed equally to this work; (5) Donghyun Na, Beeble AI, and contributed equally to this work; (6) Sanghyun Woo, New York University, and contributed equally to this work. Authors: Authors: (1) Hoon Kim, Beeble AI, and contributed equally to this work; (2) Minje Jang, Beeble AI, and contributed equally to this work; (3) Wonjun Yoon, Beeble AI, and contributed equally to this work; (4) Jisoo Lee, Beeble AI, and contributed equally to this work; (5) Donghyun Na, Beeble AI, and contributed equally to this work; (6) Sanghyun Woo, New York University, and contributed equally to this work. Editor's Note: This is Part 2 of 14 of a study introducing a method for improving how light and shadows can be applied to human portraits in digital images. Read the rest below. Editor's Note: This is Part 2 of 14 of a study introducing a method for improving how light and shadows can be applied to human portraits in digital images. Read the rest below. Editor's Note: This is Part 2 of 14 of a study introducing a method for improving how light and shadows can be applied to human portraits in digital images. Read the rest below. Editor's Note: This is Part 2 of 14 of a study introducing a method for improving how light and shadows can be applied to human portraits in digital images. Read the rest below. Table of Links Abstract and 1. Introduction 2. Related Work 3. SwitchLight and 3.1. Preliminaries 3.2. Problem Formulation 3.3. Architecture 3.4. Objectives 4. Multi-Masked Autoencoder Pre-training 5. Data 6. Experiments 7. Conclusion Abstract and 1. Introduction Abstract and 1. Introduction 2. Related Work 2. Related Work 3. SwitchLight and 3.1. Preliminaries 3. SwitchLight and 3.1. Preliminaries 3.2. Problem Formulation 3.2. Problem Formulation 3.3. Architecture 3.3. Architecture 3.4. Objectives 3.4. Objectives 4. Multi-Masked Autoencoder Pre-training 4. Multi-Masked Autoencoder Pre-training 5. Data 5. Data 6. Experiments 6. Experiments 7. Conclusion 7. Conclusion Appendix A. Implementation Details B. User Study Interface C. Video Demonstration D. Additional Qualitative Results & References A. Implementation Details A. Implementation Details B. User Study Interface B. User Study Interface C. Video Demonstration C. Video Demonstration D. Additional Qualitative Results & References D. Additional Qualitative Results & References 2. Related Work Human Portrait Relighting is an ill-posed problem due to its under-constrained nature. To tackle this, earlier methods incorporated 3D facial priors [44], exploited image intrinsics [3, 40], or framed the task as a style transfer [43]. Light stage techniques [49] offer a more powerful solution by recording subject’s reflectance fields under varying lighting conditions [10, 14], though they are labor-intensive and require specialized equipment. A promising alternative has emerged with deep learning, utilizing neural networks trained on light stage data. Sun et al. [45] pioneered this approach, but their method had limitations in representing non-Lambertian effects. This was improved upon by Nestmeyer et al. [32], who integrated rendering physics into network design, albeit limited to directional light. Building upon this, Pandey et al. [34] incorporated the Phong reflection model and a high dynamic range (HDR) lighting map [9] into their network, enabling a more accurate representation of global illumination. Simultaneously, efforts have been made to explore portrait relighting without light stage data [20, 21, 42, 47, 55]. Moreover, introduction of NeRF [7] and diffusion-based [38] models has opened new avenues in the field. However, networks trained with lightstage data maintain superior accuracy and realism, thanks to physics-based composited relight image training pairs and precise ground truth image intrinsics [56]. Human Portrait Relighting Our work furthers this domain by integrating the Cook-Torrance model into our network design, shifting from the empirical Phong model to a more physics-based approach, thereby enhancing the realism and detail in relit images. Self-supervised Pre-training has become a standard training scheme in the development of large language models like BERT [11] and GPT [39], and is increasingly influential in vision models, aiming to replicate the ‘BERT moment’. This approach typically involves pre-training on extensive unlabeled data, followed by fine-tuning on specific tasks. While early efforts in vision models focused on simple pretext tasks [13, 17, 33, 36, 53], the field has evolved through stages like contrastive learning [5, 18] and masked image modeling [2, 19, 50]. However, the primary focus has remained on visual recognition, with less attention to other domains. Exceptions include low-level image processing tasks [4, 6, 27, 30] using the vision transformer [15]. Self-supervised Pre-training Our research takes a different route, focusing on human portrait relighting—a complex challenge of manipulating illumination in the image. This direction is crucial because acquiring accurate ground truth data, especially from light stage, is both expensive and difficult. We modify the MAE framework [19], previously successful in robust image representation learning and developing locality biases [35], to suit the unique requirements of effective relighting. This paper is available on arxiv under CC BY-NC-SA 4.0 DEED license. This paper is available on arxiv under CC BY-NC-SA 4.0 DEED license. available on arxiv

Beeble Researchers Use Physics-Based Models to Achieve Realistic Lighting Effects in Images

Table of Links Abstract and 1 Introduction 2 Related Work 2.1 Neural Codec Language Models and 2.2 Non-autoregressive Models 2.3 Diffusion Models and 2.4 Zero-shot Voice Cloning 3 Hierspeech++ and 3.1 Speech Representations 3.2 Hierarchical Speech Synthesizer 3.3 Text-to-Vec 3.4 Speech Super-resolution 3.5 Model Architecture 4 Speech Synthesis Tasks 4.1 Voice Conversion and 4.2 Text-to-Speech 4.3 Style Prompt Replication 5 Experiment and Result, and Dataset 5.2 Preprocessing and 5.3 Training 5.4 Evaluation Metrics 5.5 Ablation Study 5.6 Zero-shot Voice Conversion 5.7 High-diversity but High-fidelity Speech Synthesis 5.8 Zero-shot Text-to-Speech 5.9 Zero-shot Text-to-Speech with 1s Prompt 5.10 Speech Super-resolution 5.11 Additional Experiments with Other Baselines 6 Limitation and Quick Fix 7 Conclusion, Acknowledgement and References 5.5 Ablation Study Before we compare the model with other baselines in the TTS and VC tasks, we conducted ablation studies by comparing Reconstruction[10], Resynthesis [11], and VC performance to verify the effectiveness of each component in HierSpeech++. First, although previous E2E models have shown highquality waveform audio generation, the zero-shot speech synthesis performance was considerably low, and some studies must fine-tune or use speaker id for speaker adaptation. Recently, HierVST has significantly improved a voice style transfer performance of the E2E model; therefore so we conduct ablation studies by building up on HierVST AMP Block We first replaced the MRF block of HiFi-GAN with the AMP of BigVGAN for OOD generation. The AMP improved the performance of all tasks in terms of all metrics without F0 consistency. The results show that the BigVGANbased HAG performs better but the loss balance may lean toward optimizing the waveform reconstruction rather than F0 prediction; however, the naturalness and similarity of the converted speech improved in terms of all metrics. Specifically, objective naturalness exhibited better UTMOS. SF Encoder To address F0 consistency, we utilize an SF encoder (SFE) for a dual-path semantic encoder, which enhances the semantic prior in terms of all metrics. This significantly improved the F0 consistency of inference scenario. It is worth noting that F0 can be manually controlled. Dual-audio Encoder We also utilized a dual-audio posterior encoder (DAE) to increase the acoustic capacity of the acoustic representation, which significantly increases the reconstruction performance. Although the linear spectrogram contains useful information for reconstructing a waveform audio, this representation still lacks the ability to reproduce all information; therefore, additional information from waveform audio could complement a wave-level acoustic representation. It is worth noting that the DAE was only utilized during training but significantly improved reconstruction and pronunciation. However, we found that the enhanced acoustic posterior contains a large information resulting in reducing a VC performance. T-Flow To bridge the gap between each representation, we replace a wavenet-based normalizing flow with Transformerbased normalizing flow (T-Flow) using AdaLN-Zero for style adaptation. This also improved the entire performance of all metrics. Moreover, speaker similarity significantly improved. Bi-Flow Moreover, we adopt a bidirectional normalizing flow (Bi-Flow) to reduce the train-inference mismatch problem. The results show that Bi-Flow slightly decreases the reconstruction quality. However, this could regularize a posterior by conditioning the information used in the inference scenario, thereby improving the VC performance. We also found that high weight of Bi-Flow significantly decreased a reconstruction performance, and thus, we use λ of 0.5 for weak regularization. Large-scale Data In addition, we demonstrated that our model is robust to data scale-up. We did not use any labels to train the model, and only used a low-resolution speech dataset of 16 kHz, which we could obtain simply due to SpeechSR. For scaling-up, we did not conduct any preprocessing to train the model without down-sampling (any to 16 kHz), so there are noisy samples in our dataset however we did not experience any problems. This paper is available on arxiv under CC BY-NC-SA 4.0 DEED license. [10] Reconstruction: Posterior Encoder → Generator → Audio [11] Resynthesis: Prior Encoder → Generator → Audio Authors:
(1) Sang-Hoon Lee, Fellow, IEEE with the Department of Artificial Intelligence, Korea University, Seoul 02841, South Korea;
(2) Ha-Yeong Choi, Fellow, IEEE with the Department of Artificial Intelligence, Korea University, Seoul 02841, South Korea;
(3) Seung-Bin Kim, Fellow, IEEE with the Department of Artificial Intelligence, Korea University, Seoul 02841, South Korea;
(4) Seong-Whan Lee, Fellow, IEEE with the Department of Artificial Intelligence, Korea University, Seoul 02841, South Korea and a Corresponding author. Table of Links Abstract and 1 Introduction Abstract and 1 Introduction 2 Related Work 2.1 Neural Codec Language Models and 2.2 Non-autoregressive Models 2.1 Neural Codec Language Models and 2.2 Non-autoregressive Models 2.3 Diffusion Models and 2.4 Zero-shot Voice Cloning 2.3 Diffusion Models and 2.4 Zero-shot Voice Cloning 3 Hierspeech++ and 3.1 Speech Representations 3 Hierspeech++ and 3.1 Speech Representations 3.2 Hierarchical Speech Synthesizer 3.2 Hierarchical Speech Synthesizer 3.3 Text-to-Vec 3.3 Text-to-Vec 3.4 Speech Super-resolution 3.4 Speech Super-resolution 3.5 Model Architecture 3.5 Model Architecture 4 Speech Synthesis Tasks 4.1 Voice Conversion and 4.2 Text-to-Speech 4.1 Voice Conversion and 4.2 Text-to-Speech 4.3 Style Prompt Replication 4.3 Style Prompt Replication 5 Experiment and Result, and Dataset 5 Experiment and Result, and Dataset 5.2 Preprocessing and 5.3 Training 5.2 Preprocessing and 5.3 Training 5.4 Evaluation Metrics 5.4 Evaluation Metrics 5.5 Ablation Study 5.5 Ablation Study 5.6 Zero-shot Voice Conversion 5.6 Zero-shot Voice Conversion 5.7 High-diversity but High-fidelity Speech Synthesis 5.7 High-diversity but High-fidelity Speech Synthesis 5.8 Zero-shot Text-to-Speech 5.8 Zero-shot Text-to-Speech 5.9 Zero-shot Text-to-Speech with 1s Prompt 5.9 Zero-shot Text-to-Speech with 1s Prompt 5.10 Speech Super-resolution 5.10 Speech Super-resolution 5.11 Additional Experiments with Other Baselines 5.11 Additional Experiments with Other Baselines 6 Limitation and Quick Fix 6 Limitation and Quick Fix 7 Conclusion, Acknowledgement and References 7 Conclusion, Acknowledgement and References 5.5 Ablation Study Before we compare the model with other baselines in the TTS and VC tasks, we conducted ablation studies by comparing Reconstruction[10], Resynthesis [11], and VC performance to verify the effectiveness of each component in HierSpeech++. First, although previous E2E models have shown highquality waveform audio generation, the zero-shot speech synthesis performance was considerably low, and some studies must fine-tune or use speaker id for speaker adaptation. Recently, HierVST has significantly improved a voice style transfer performance of the E2E model; therefore so we conduct ablation studies by building up on HierVST AMP Block We first replaced the MRF block of HiFi-GAN with the AMP of BigVGAN for OOD generation. The AMP improved the performance of all tasks in terms of all metrics without F0 consistency. The results show that the BigVGANbased HAG performs better but the loss balance may lean toward optimizing the waveform reconstruction rather than F0 prediction; however, the naturalness and similarity of the converted speech improved in terms of all metrics. Specifically, objective naturalness exhibited better UTMOS. AMP Block SF Encoder To address F0 consistency, we utilize an SF encoder (SFE) for a dual-path semantic encoder, which enhances the semantic prior in terms of all metrics. This significantly improved the F0 consistency of inference scenario. It is worth noting that F0 can be manually controlled. SF Encoder Dual-audio Encoder We also utilized a dual-audio posterior encoder (DAE) to increase the acoustic capacity of the acoustic representation, which significantly increases the reconstruction performance. Although the linear spectrogram contains useful information for reconstructing a waveform audio, this representation still lacks the ability to reproduce all information; therefore, additional information from waveform audio could complement a wave-level acoustic representation. It is worth noting that the DAE was only utilized during training but significantly improved reconstruction and pronunciation. However, we found that the enhanced acoustic posterior contains a large information resulting in reducing a VC performance. Dual-audio Encoder T-Flow To bridge the gap between each representation, we replace a wavenet-based normalizing flow with Transformerbased normalizing flow (T-Flow) using AdaLN-Zero for style adaptation. This also improved the entire performance of all metrics. Moreover, speaker similarity significantly improved. T-Flow Bi-Flow Moreover, we adopt a bidirectional normalizing flow (Bi-Flow) to reduce the train-inference mismatch problem. The results show that Bi-Flow slightly decreases the reconstruction quality. However, this could regularize a posterior by conditioning the information used in the inference scenario, thereby improving the VC performance. We also found that high weight of Bi-Flow significantly decreased a reconstruction performance, and thus, we use λ of 0.5 for weak regularization. Bi-Flow Large-scale Data In addition, we demonstrated that our model is robust to data scale-up. We did not use any labels to train the model, and only used a low-resolution speech dataset of 16 kHz, which we could obtain simply due to SpeechSR. For scaling-up, we did not conduct any preprocessing to train the model without down-sampling (any to 16 kHz), so there are noisy samples in our dataset however we did not experience any problems. Large-scale Data This paper is available on arxiv under CC BY-NC-SA 4.0 DEED license. This paper is available on arxiv under CC BY-NC-SA 4.0 DEED license. available on arxiv [10] Reconstruction: Posterior Encoder → Generator → Audio [11] Resynthesis: Prior Encoder → Generator → Audio Authors: (1) Sang-Hoon Lee, Fellow, IEEE with the Department of Artificial Intelligence, Korea University, Seoul 02841, South Korea; (2) Ha-Yeong Choi, Fellow, IEEE with the Department of Artificial Intelligence, Korea University, Seoul 02841, South Korea; (3) Seung-Bin Kim, Fellow, IEEE with the Department of Artificial Intelligence, Korea University, Seoul 02841, South Korea; (4) Seong-Whan Lee, Fellow, IEEE with the Department of Artificial Intelligence, Korea University, Seoul 02841, South Korea and a Corresponding author. Authors: Authors: (1) Sang-Hoon Lee, Fellow, IEEE with the Department of Artificial Intelligence, Korea University, Seoul 02841, South Korea; (2) Ha-Yeong Choi, Fellow, IEEE with the Department of Artificial Intelligence, Korea University, Seoul 02841, South Korea; (3) Seung-Bin Kim, Fellow, IEEE with the Department of Artificial Intelligence, Korea University, Seoul 02841, South Korea; (4) Seong-Whan Lee, Fellow, IEEE with the Department of Artificial Intelligence, Korea University, Seoul 02841, South Korea and a Corresponding author.

Conducting Ablation Studies to Verify the Effectiveness of Each Component in HierSpeech++

How Effective Are AI Models at Biomedical Problem Solving?

In today's visually-driven world, users often need to expand images for purposes like enhancing social media posts, expanding promotional materials, and uncropping e-commerce product images. Conventional methods carry the risk of producing unnatural images that do not satisfy user requirements. Uncrop.org is a powerful AI tool designed to offer a more straightforward and efficient solution to uncrop photos. In this detailed review, we will explore the technology behind Uncrop.org, its unique features, and various practical applications. Join us as we uncover how Uncrop.org can elevate your image editing experience. What Is Uncrop.org? Uncrop.org is a powerful AI-driven tool designed for image extension, enabling users to outpaint images by adding high-quality, AI-generated content to the expanded areas. Leveraging advanced algorithms, the tool analyzes the provided image and intelligently predicts how to extend it, seamlessly enlarging the image's boundaries while maintaining the integrity of the original content. Uncrop.org meticulously processes your images, preserving their original quality and visual appeal. This enables you to effortlessly outpaint portraits, landscapes, product photos, and more to meet specific dimensions or aesthetic requirements. The platform ensures consistently high-quality and visually pleasing results, making it versatile for a wide range of image types. Uncrop.org is designed with ease of use in mind, catering to both novice and experienced image editors. Optimized for mobile devices, the platform processes images quickly based on their size and complexity. It also offers free uncropping and previewing features, ensuring a smooth and efficient experience for all users. How to Navigate Uncrop.org Uncrop.org provides a user-friendly platform designed to streamline the image uncropping process, catering to both beginners and experienced users alike. To start, just adhere to the steps described below: Step 1: Simply upload or drag the photo you want to process onto Uncrop.org. Step 2: Once the upload is complete, you can select your desired image size and aspect ratio. Step 3: Within minutes, the expander will produce an uncropped version of your image, ready for you to download and save. Why Should You Choose Uncrop.org? Several factors make Uncrop.org a leading AI extension tool in the market. We will explore these key aspects in detail below. Fast Image Extension Uncrop.org utilizes cutting-edge AI algorithms to provide quick and effective image extension. It can usually extend your images within minutes while maintaining high quality, making it perfect for handling extensive image editing tasks. Extensive Image Format Compatibility With Uncrop.org, you can upload various image formats without the need for prior file conversion. The platform supports JPG, PNG, and even WebP formats, handling images up to 50MB in size. This ensures smooth compatibility for outpainting your photos. Limitless Image Expansion Uncrop.org offers you unrestricted capabilities to extend your images. You can easily resize and adjust the aspect ratio to your liking with just a few clicks. This allows you to uncrop your image precisely as you envision. Effortless AI Photo Extension Uncrop.org works by analyzing the elements of an image to seamlessly extend it, ensuring the new sections blend perfectly with the original. This leads to natural and contextually accurate extensions that consistently prevent any image distortion. In What Ways Can You Make Use of Uncrop.org? Uncrop.org stands out as a highly flexible AI tool designed for outpainting images, making it valuable for numerous applications across diverse sectors, industries, and professional areas. Its utility shines in many typical situations, including but not limited to the following: Create for Social Media: To boost engagement on social media through visually captivating images, having flexibility with aspect ratios is essential. Each social media platform has its unique way of displaying images, so adjusting them accordingly is important. Uncrop.org enables you to tailor your photos to meet the specific dimensions needed for each platform, ensuring that your visuals are ideally suited for every social media site. Outpaint Advertising Materials: When creating promotional materials like posters and banners, it’s often essential to resize images to match various dimensions. Uncrop.org allows you to extend your visuals, ensuring they perfectly fit different advertising formats without losing their visual appeal or key message. This ensures consistent and visually appealing promotional content across multiple platforms. Uncrop Photographs & Artworks: Using Uncrop.org, you can seamlessly expand any photograph or artwork, enriching your visual storytelling and giving your audience a deeper experience of the narrative and composition. This tool is especially advantageous for crafting presentations or displays that thrive on larger, more engaging visuals. Extend Product Images: Making sure that product images have the appropriate dimensions and sizes is essential for attracting potential online customers. With Uncrop.org, you can easily extend your product photos and adjust their dimensions to boost their visual appeal. The tool is crafted to preserve picture quality with minimal distortion, ensuring that the images stay clear and attractive. To Sum Up Uncrop.org provides an outstanding solution for those looking to instantly enlarge images with seamless, naturally extended content. Leveraging superior AI technology, it guarantees smooth and visually appealing expansions, no matter the size requirement. Plus, it delivers results swiftly and is available at no cost. Try out Uncrop.org now! In today's visually-driven world, users often need to expand images for purposes like enhancing social media posts, expanding promotional materials, and uncropping e-commerce product images. Conventional methods carry the risk of producing unnatural images that do not satisfy user requirements. Uncrop.org is a powerful AI tool designed to offer a more straightforward and efficient solution to uncrop photos. In this detailed review, we will explore the technology behind Uncrop.org, its unique features, and various practical applications. Join us as we uncover how Uncrop.org can elevate your image editing experience. Uncrop.org Uncrop.org What Is Uncrop.org? Uncrop.org is a powerful AI-driven tool designed for image extension, enabling users to outpaint images by adding high-quality, AI-generated content to the expanded areas. Leveraging advanced algorithms, the tool analyzes the provided image and intelligently predicts how to extend it, seamlessly enlarging the image's boundaries while maintaining the integrity of the original content. Uncrop.org meticulously processes your images, preserving their original quality and visual appeal. This enables you to effortlessly outpaint portraits, landscapes, product photos, and more to meet specific dimensions or aesthetic requirements. The platform ensures consistently high-quality and visually pleasing results, making it versatile for a wide range of image types. Uncrop.org is designed with ease of use in mind, catering to both novice and experienced image editors. Optimized for mobile devices, the platform processes images quickly based on their size and complexity. It also offers free uncropping and previewing features, ensuring a smooth and efficient experience for all users. How to Navigate Uncrop.org Uncrop.org provides a user-friendly platform designed to streamline the image uncropping process, catering to both beginners and experienced users alike. To start, just adhere to the steps described below: Step 1: Simply upload or drag the photo you want to process onto Uncrop.org . Uncrop.org Uncrop.org Step 2: Once the upload is complete, you can select your desired image size and aspect ratio. Step 3: Within minutes, the expander will produce an uncropped version of your image, ready for you to download and save. Why Should You Choose Uncrop.org? Several factors make Uncrop.org a leading AI extension tool in the market. We will explore these key aspects in detail below. Fast Image Extension Uncrop.org utilizes cutting-edge AI algorithms to provide quick and effective image extension. It can usually extend your images within minutes while maintaining high quality, making it perfect for handling extensive image editing tasks. Extensive Image Format Compatibility With Uncrop.org, you can upload various image formats without the need for prior file conversion. The platform supports JPG, PNG, and even WebP formats, handling images up to 50MB in size. This ensures smooth compatibility for outpainting your photos. Limitless Image Expansion Uncrop.org offers you unrestricted capabilities to extend your images. You can easily resize and adjust the aspect ratio to your liking with just a few clicks. This allows you to uncrop your image precisely as you envision. Effortless AI Photo Extension Uncrop.org works by analyzing the elements of an image to seamlessly extend it, ensuring the new sections blend perfectly with the original. This leads to natural and contextually accurate extensions that consistently prevent any image distortion. In What Ways Can You Make Use of Uncrop.org? Uncrop.org stands out as a highly flexible AI tool designed for outpainting images, making it valuable for numerous applications across diverse sectors, industries, and professional areas. Its utility shines in many typical situations, including but not limited to the following: Create for Social Media: To boost engagement on social media through visually captivating images, having flexibility with aspect ratios is essential. Each social media platform has its unique way of displaying images, so adjusting them accordingly is important. Uncrop.org enables you to tailor your photos to meet the specific dimensions needed for each platform, ensuring that your visuals are ideally suited for every social media site. Create for Social Media: To boost engagement on social media through visually captivating images, having flexibility with aspect ratios is essential. Each social media platform has its unique way of displaying images, so adjusting them accordingly is important. Uncrop.org enables you to tailor your photos to meet the specific dimensions needed for each platform, ensuring that your visuals are ideally suited for every social media site. Outpaint Advertising Materials: When creating promotional materials like posters and banners, it’s often essential to resize images to match various dimensions. Uncrop.org allows you to extend your visuals, ensuring they perfectly fit different advertising formats without losing their visual appeal or key message. This ensures consistent and visually appealing promotional content across multiple platforms. Outpaint Advertising Materials: When creating promotional materials like posters and banners, it’s often essential to resize images to match various dimensions. Uncrop.org allows you to extend your visuals, ensuring they perfectly fit different advertising formats without losing their visual appeal or key message. This ensures consistent and visually appealing promotional content across multiple platforms. Uncrop Photographs & Artworks: Using Uncrop.org, you can seamlessly expand any photograph or artwork, enriching your visual storytelling and giving your audience a deeper experience of the narrative and composition. This tool is especially advantageous for crafting presentations or displays that thrive on larger, more engaging visuals. Uncrop Photographs & Artworks: Using Uncrop.org, you can seamlessly expand any photograph or artwork, enriching your visual storytelling and giving your audience a deeper experience of the narrative and composition. This tool is especially advantageous for crafting presentations or displays that thrive on larger, more engaging visuals. Extend Product Images: Making sure that product images have the appropriate dimensions and sizes is essential for attracting potential online customers. With Uncrop.org, you can easily extend your product photos and adjust their dimensions to boost their visual appeal. The tool is crafted to preserve picture quality with minimal distortion, ensuring that the images stay clear and attractive. Extend Product Images: Making sure that product images have the appropriate dimensions and sizes is essential for attracting potential online customers. With Uncrop.org, you can easily extend your product photos and adjust their dimensions to boost their visual appeal. The tool is crafted to preserve picture quality with minimal distortion, ensuring that the images stay clear and attractive. To Sum Up Uncrop.org provides an outstanding solution for those looking to instantly enlarge images with seamless, naturally extended content. Leveraging superior AI technology, it guarantees smooth and visually appealing expansions, no matter the size requirement. Plus, it delivers results swiftly and is available at no cost. Try out Uncrop.org now!

Uncrop.org Review: Premier AI Expander for Outpainting Images Effectively

Uncrop.org recenzija: Premier AI Expander za efikasno preslikavanje slika

Reseña de Uncrop.org: el mejor expansor de imágenes con inteligencia artificial para pintar imágenes de manera eficaz

Ulasan Uncrop.org: Pengembang AI Terkemuka untuk Menggambar Gambar Secara Efektif

Recensione di Uncrop.org: Premier AI Expander per dipingere efficacemente le immagini

Uncrop.org レビュー: 画像を効果的に塗りつぶすプレミア AI エクスパンダー

Uncrop.org Review: Premier AI Expander voor het effectief overschilderen van afbeeldingen

Uncrop.org Rikuy: Premier AI mast'ariq siq'ikunata allinta pintanapaq

Uncrop.org Ongororo: Premier AI Expander yeKubuda Kwemifananidzo Zvinobudirira

Uncrop.org விமர்சனம்: படங்களைத் திறம்பட அவுட் பெயிண்டிங் செய்வதற்கான பிரீமியர் AI எக்ஸ்பாண்டர்

รีวิว Uncrop.org: โปรแกรม Premier AI Expander สำหรับระบายสีภาพอย่างมีประสิทธิภาพ

Pagsusuri sa Uncrop.org: Premier AI Expander para sa Mabisang Pag-outpainting ng mga Larawan

Uncrop.org İncelemesi: Görüntüleri Etkili Şekilde Boyamak İçin Birinci Sınıf AI Genişletici

JavaScript challenges are like stealthy ninjas lurking in the shadows 🌃, ready to block your web scraping efforts without you even realizing it. They may not be visible, but their presence can thwart your data collection attempts! Dig into how these challenges operate and explore effective strategies for bypassing them. Time to enhance your web scraping capabilities! 🦾 What Are JavaScript Challenges? Nope, we’re not talking about those fun JavaScript coding challenges we all love. That’s a whole different game... Here, we’re exploring a different type of challenge. 🤔 In the world of bot protection, JavaScript challenges—also known as JS challenges—are the digital bouncers that stand between your scraper and a page's juicy content. They’re there to keep automated scraping bots from accessing a site's data. 🚫 🤖 🚫 Web servers embed these challenges directly into the web pages they deliver to the client. To bypass them and access the site’s content, you need a browser that can execute the JavaScript code within these challenge scripts. Otherwise, you’re not getting in! 🛑 Sites use the JavaScript challenge mechanism to automatically detect and block bots. Think of it as a “prove you’re human” test. To gain entry to the site, your scraper must be able to run some specific obfuscated script in a browser and pass the underlying test! What Does a JavaScript Challenge Look Like? Usually, a JavaScript challenge is like a ghost 👻—you can sense it, but you rarely see it. More specifically, it’s just a script hiding in the web page that your browser must execute to gain access to the site’s content. To get a clearer picture of these challenges, let’s look at a real-world example. Cloudflare is known for using JS challenges. When you enable the Managed Challenge feature of its WAF (Web Application Firewall) solution, the popular CDN starts embedding JavaScript challenges in your pages. According to official docs, a JS challenge doesn’t require user interaction. Instead, it’s processed quietly by the browser in the background. ⚙️ During this process, the JavaScript code runs tests to confirm if the visitor is human👤—like checking for the presence of specific fonts installed on the user’s device. In detail, Cloudflare uses Google’s Picasso fingerprinting protocol. This analyzes the client’s software and hardware stack with data collected via JavaScript. The entire verification process might happen behind the scenes without the user noticing, or it might stall them briefly with a screen like this: Want to avoid this screen altogether? Read the guide on Cloudflare bypass! Now, three scenarios can play out: You pass the test: You access the page, and the JavaScript challenge won’t reappear during the same browsing session.
You fail the test: Expect to face additional anti-bot measures, like CAPTCHAs.
You can’t run the test: If you’re using an HTTP client that can’t execute JavaScript, you’re out of luck—blocked, and possibly banned! (Pro tip: Learn how to avoid IP bans with proxies!). How to Challenge JavaScript Protections for Seamless Web Scraping Want to bypass mandatory JavaScript challenges? First, you need an automation tool that runs web pages in a browser 🌐. In other words, you have to use a browser automation library like Selenium, Puppeteer, or Playwright. Those tools empower you to write scraping scripts that make a real browser interact with web pages just like a human would. This strategy helps you bypass the dreaded scenario 3 (you can’t run the test) from earlier, limiting your outcomes to either scenario 1 (you pass the test) or scenario 2 (you fail the test). For simple JavaScript challenges that just check if you can run JS, a browser automation tool is usually enough to do the trick 😌. But when it comes to more advanced challenges from services like Cloudflare or Akamai, things get tricky… To control browsers, these tools set configurations that can raise suspicion with WAFs. You can try to hide them using technologies like Puppeteer Extra, but that doesn’t always guarantee success either. 🥷 Suspicious settings are especially evident when checking browsers in headless mode, which is popular in scraping due to its resource efficiency. However, don’t forget that headless browsers are still resource-intensive compared to HTTP clients. So, they require a solid server setup to run at scale. ⚖️ So, what’s the ultimate answer for overcoming JavaScript challenges and doing scraping without getting blocked and at scale? Best Solution to Overcome a JavaScript Challenge The issue isn't with the browser automation tools themselves. Quite the opposite, it’s all about the browsers those solutions control! 💡 Now, picture a browser that: Runs in headed mode like a regular browser, reducing the chances of bot detection.


Scales effortlessly in the cloud, saving you both time and money on infrastructure management.


Automatically tackles CAPTCHA solving, browser fingerprinting, cookie and header customization, and retries for optimal efficiency.


Provides rotating IPs backed by one of the largest and most reliable proxy networks out there.


Seamlessly integrates with popular browser automation libraries like Playwright, Selenium, and Puppeteer. If such a solution existed, it would allow you to wave goodbye to JavaScript challenges and most other anti-scraping measures. Well, this isn’t just a distant fantasy—it's a reality! Enter Bright Data’s Scraping Browser: https://www.youtube.com/watch?v=kuDuJWvho7Q&embedable=true Final Thoughts Now you’re in the loop about JavaScript challenges and why they’re not just tests to level up your coding skills. In the realm of web scraping, these challenges are pesky barriers that can stop your data retrieval efforts. Want to scrape without hitting those frustrating blocks? Take a look at Bright Data's suite of tools! Join our mission to make the Internet accessible to everyone—even via automated browsers. 🌐 Until next time, keep surfing the Internet with freedom! JavaScript challenges are like stealthy ninjas lurking in the shadows 🌃, ready to block your web scraping efforts without you even realizing it. They may not be visible, but their presence can thwart your data collection attempts! Dig into how these challenges operate and explore effective strategies for bypassing them. Time to enhance your web scraping capabilities! 🦾 What Are JavaScript Challenges? Nope, we’re not talking about those fun JavaScript coding challenges we all love. That’s a whole different game... Here, we’re exploring a different type of challenge. 🤔 In the world of bot protection, JavaScript challenges—also known as JS challenges—are the digital bouncers that stand between your scraper and a page's juicy content. They’re there to keep automated scraping bots from accessing a site's data. 🚫 🤖 🚫 scraping bots Web servers embed these challenges directly into the web pages they deliver to the client. To bypass them and access the site’s content, you need a browser that can execute the JavaScript code within these challenge scripts. Otherwise, you’re not getting in! 🛑 Sites use the JavaScript challenge mechanism to automatically detect and block bots. Think of it as a “prove you’re human” test. To gain entry to the site, your scraper must be able to run some specific obfuscated script in a browser and pass the underlying test! What Does a JavaScript Challenge Look Like? Usually, a JavaScript challenge is like a ghost 👻—you can sense it, but you rarely see it. More specifically, it’s just a script hiding in the web page that your browser must execute to gain access to the site’s content. To get a clearer picture of these challenges, let’s look at a real-world example. Cloudflare is known for using JS challenges. When you enable the Managed Challenge feature of its WAF ( Web Application Firewall ) solution, the popular CDN starts embedding JavaScript challenges in your pages. Managed Challenge Web Application Firewall According to official docs, a JS challenge doesn’t require user interaction. Instead, it’s processed quietly by the browser in the background. ⚙️ JS challenge During this process, the JavaScript code runs tests to confirm if the visitor is human👤—like checking for the presence of specific fonts installed on the user’s device. In detail, Cloudflare uses Google’s Picasso fingerprinting protocol . This analyzes the client’s software and hardware stack with data collected via JavaScript. Google’s Picasso fingerprinting protocol The entire verification process might happen behind the scenes without the user noticing, or it might stall them briefly with a screen like this: Want to avoid this screen altogether? Read the guide on Cloudflare bypass ! Cloudflare bypass Now, three scenarios can play out: You pass the test: You access the page, and the JavaScript challenge won’t reappear during the same browsing session. You fail the test: Expect to face additional anti-bot measures, like CAPTCHAs. You can’t run the test: If you’re using an HTTP client that can’t execute JavaScript, you’re out of luck—blocked, and possibly banned! (Pro tip: Learn how to avoid IP bans with proxies!). You pass the test : You access the page, and the JavaScript challenge won’t reappear during the same browsing session. You pass the test You fail the test : Expect to face additional anti-bot measures, like CAPTCHAs. You fail the test You can’t run the test : If you’re using an HTTP client that can’t execute JavaScript, you’re out of luck—blocked, and possibly banned! (Pro tip: Learn how to avoid IP bans with proxies! ). You can’t run the test Learn how to avoid IP bans with proxies! How to Challenge JavaScript Protections for Seamless Web Scraping Want to bypass mandatory JavaScript challenges? First, you need an automation tool that runs web pages in a browser 🌐. In other words, you have to use a browser automation library like Selenium, Puppeteer, or Playwright . browser automation library like Selenium, Puppeteer, or Playwright Those tools empower you to write scraping scripts that make a real browser interact with web pages just like a human would. This strategy helps you bypass the dreaded scenario 3 (you can’t run the test) from earlier, limiting your outcomes to either scenario 1 (you pass the test) or scenario 2 (you fail the test). For simple JavaScript challenges that just check if you can run JS, a browser automation tool is usually enough to do the trick 😌. But when it comes to more advanced challenges from services like Cloudflare or Akamai, things get tricky… To control browsers, these tools set configurations that can raise suspicion with WAFs. You can try to hide them using technologies like Puppeteer Extra , but that doesn’t always guarantee success either. 🥷 Puppeteer Extra Suspicious settings are especially evident when checking browsers in headless mode, which is popular in scraping due to its resource efficiency. However, don’t forget that headless browsers are still resource-intensive compared to HTTP clients. So, they require a solid server setup to run at scale. ⚖️ So, what’s the ultimate answer for overcoming JavaScript challenges and doing scraping without getting blocked and at scale ? scraping without getting blocked and at scale Best Solution to Overcome a JavaScript Challenge The issue isn't with the browser automation tools themselves. Quite the opposite, it’s all about the browsers those solutions control! 💡 Now, picture a browser that: Runs in headed mode like a regular browser, reducing the chances of bot detection. Scales effortlessly in the cloud, saving you both time and money on infrastructure management. Automatically tackles CAPTCHA solving, browser fingerprinting, cookie and header customization, and retries for optimal efficiency. Provides rotating IPs backed by one of the largest and most reliable proxy networks out there. Seamlessly integrates with popular browser automation libraries like Playwright, Selenium, and Puppeteer. Runs in headed mode like a regular browser, reducing the chances of bot detection. Runs in headed mode like a regular browser, reducing the chances of bot detection. Scales effortlessly in the cloud, saving you both time and money on infrastructure management. Scales effortlessly in the cloud, saving you both time and money on infrastructure management. Automatically tackles CAPTCHA solving, browser fingerprinting, cookie and header customization, and retries for optimal efficiency. Automatically tackles CAPTCHA solving, browser fingerprinting, cookie and header customization, and retries for optimal efficiency. Provides rotating IPs backed by one of the largest and most reliable proxy networks out there. Provides rotating IPs backed by one of the largest and most reliable proxy networks out there. Seamlessly integrates with popular browser automation libraries like Playwright, Selenium, and Puppeteer. Seamlessly integrates with popular browser automation libraries like Playwright, Selenium, and Puppeteer. If such a solution existed, it would allow you to wave goodbye to JavaScript challenges and most other anti-scraping measures . Well, this isn’t just a distant fantasy—it's a reality! anti-scraping measures Enter Bright Data’s Scraping Browser : Scraping Browser https://www.youtube.com/watch?v=kuDuJWvho7Q&embedable=true https://www.youtube.com/watch?v=kuDuJWvho7Q&embedable=true Final Thoughts Now you’re in the loop about JavaScript challenges and why they’re not just tests to level up your coding skills. In the realm of web scraping, these challenges are pesky barriers that can stop your data retrieval efforts. Want to scrape without hitting those frustrating blocks? Take a look at Bright Data 's suite of tools! Join our mission to make the Internet accessible to everyone—even via automated browsers. 🌐 Bright Data Until next time, keep surfing the Internet with freedom!

Bypassing JavaScript Challenges for Effective Web Scraping

تجاوز تحديات JavaScript لتحقيق كفاءة في استخراج البيانات من الويب

কার্যকর ওয়েব স্ক্র্যাপিংয়ের জন্য জাভাস্ক্রিপ্ট চ্যালেঞ্জগুলিকে বাইপাস করা

Παράκαμψη των προκλήσεων JavaScript για αποτελεσματική απόξεση ιστού

Cómo evitar los desafíos de JavaScript para un raspado web eficaz

دور زدن چالش های جاوا اسکریپت برای اسکرپینگ موثر وب

効果的なウェブスクレイピングのための JavaScript の課題を回避する

Обход проблем JavaScript для эффективного веб-скрапинга

Заобилажење ЈаваСцрипт изазова за ефикасно стругање веба

การหลีกเลี่ยงความท้าทายของ JavaScript เพื่อการขูดเว็บที่มีประสิทธิภาพ

Etkili Web Kazıma için JavaScript Zorluklarını Aşma

موثر ویب سکریپنگ کے لیے جاوا اسکرپٹ چیلنجز کو نظرانداز کرنا

Veb-saytlarni samarali qirqish uchun JavaScript qiyinchiliklarini chetlab o'tish

16 Open-Source Alternatives to Popular SaaS Tools: Cost-Effective & Customizable Solutions 👨‍💻🔥

A microservices architecture is the gold standard for building scalable web applications. Gartner estimates that 74% of organizations use microservices for their web applications, with another 23% planning to use them soon. If you’re an IT leader, architect, or developer, you might have experienced the faster deployments, better fault isolation, and easier scaling that come with microservices. However, the old maxim, “there’s no such thing as a free lunch,” rings true here. Because microservices can also lead to significant operational overhead and complexity. Microservices sprawl, unclear component dependencies, service duplication, and anti-patterns like circular dependencies are all part of the chaos that comes hand-in-hand with microservices. Perhaps you’ve seen some of the common symptoms of this microservices chaos: Engineering velocity drastically slows down
Onboarding new developers grows more difficult
Mean time to recovery (MTTR) for outages or performance issues increases
Overall system resiliency diminishes, putting business outcomes at risk While it’s still best practice to modernize monolithic applications by breaking them into microservices, it’s also best practice to proactively guard against the issues that microservices introduce. Architecture Governance to Control the Chaos The best way to proactively guard against these problems is by using architecture governance — a collection of rules, processes, and practices that manage and control your software architecture. With proper software architecture governance, you can reduce microservices complexity, ramp up developers faster, reduce MTTR, and improve the resiliency of your system, all while building a culture of intentionality. (Note that we’re talking about software architecture governance in this article, not enterprise architecture governance, which has been around for some time and is concerned with the policies, processes, and procedures of enterprise architecture strategies.) But exactly how do you build architecture governance? Governance, whether we’re talking about access, data, or architecture, is often just establishing a set of rules. But for architecture governance, you need more than just rules. You must start with architectural observability. Architectural Observability: The Foundation of Good Governance To implement effective governance, you first need comprehensive visibility into your software architecture. Architectural observability gives you a continuous, real-time view of your architecture. It provides your team with the necessary insights to manage the architecture and interconnectedness of distributed applications. Architectural observability provides documentation from a working system, it shows you systems flows, dependencies, what’s changing, and how your systems interact. It gives your team a thorough understanding of how the architecture works, how it drifts from release to release, and how its changes impact dependent services and resources. Without this visibility, you’re in trouble. If you’ve built a microservices project, you’ve probably encountered the results of a lack of visibility: Microservices that are difficult to manage
Unclear component dependencies
Service duplication
Anti-patterns such as circular dependencies that increase the risk of outages
Performance issues Your overall application resiliency, scalability, and engineering velocity are at high risk. And it only gets worse — the longer you let this lack of visibility continue, the more complex and unmanageable your architecture becomes. Good Observability Equals Good Governance However, with good architectural observability, you can visualize your architecture layer in real-time, gain an understanding of how your system works, and monitor changes as your architecture evolves. With observability, your team can proactively address issues, such as service sprawl and dependency mismanagement — before they lead to technical debt or system failures. And once you have that observability in place — once you know what’s happening and what’s changing — you can now create and enforce your rules of governance. You can implement a governance program that sits over your architecture, monitors, and controls changes, and allows you to build and manage microservices effectively by proactively applying standards and rules to your architecture and codebase as it evolves. Sounds great! But how do you actually build this? Let’s look at an example of how to build this in the real world. Implementing Architectural Observability and Governance First, you need to implement your architectural observability. The domain of architectural observability is new and rapidly evolving. For our example, we’ll use one of the platforms at the forefront — vFunction. vFunction is an AI-driven platform that provides visualizations of software architecture. vFunction leverages AI and OpenTelemetry tracing to enable teams to track every aspect of their architecture and monitor changes between releases. It can quickly identify complex flows, architectural, technical debt, and architectural drift. For example, vFunction allows you to better understand your distributed applications: Visualize the architecture and automatically create exportable sequence diagrams of all system and data flows.
Identify drift, such as new services or dependencies.
Identify resource exclusivity changes (such as multiple new services accessing the same database table) versus previous releases.
Identify circular dependencies between services that could impact resiliency.
Identify duplicate functionality/services that may be merged to remove complexity.
Maintain an up-to-date, real-time view of the global microservices ecosystem. vFunction uses the same data that APM tools use (so it’s straightforward to install) but uses a different layer of intelligence to address the architectural root cause of issues. This is especially important for managing the sprawl and complexity that result from the rapid and frequent deployment of microservices. Implementing Rules to Maintain Standards After implementing your observability, you’re ready to create and enforce your rules of governance. vFunction allows you to create architecture rules and tags and then monitor your architecture in real time against those rules, enforcing your critical governance policies. Any time a deployment or merge request breaks these rules, you can block that action. What type of rules might your governance enforce? You could enforce rules such as: Dependency requirements. For example, certain services should or should not communicate with other services.
Restricted resources.
Multiple services should not depend on the same database table.
Changes should not increase service interdependencies.
Restrictions on new multi-hop flows that could impact performance and resiliency. vFunction can watch and send alerts for all of the above. By implementing a focused set of rules and tags, you can create adaptive governance that guides developers in maintaining an optimal, clean, and efficient architecture. You can build architecture governance into your code and your culture. Conclusion Microservices chaos can threaten to derail development velocity, system resiliency, and operational efficiency. Architecture governance offers a way out of this chaos — a way to regain control over your architecture and steer it toward greater scalability and stability. Whether your organization is just starting to scale with microservices or is already in the depths of microservices sprawl, it’s the right time to introduce architecture governance to ensure that your architecture is efficient and resilient — and ready to support your next phase of growth. A microservices architecture is the gold standard for building scalable web applications. Gartner estimates that 74% of organizations use microservices for their web applications , with another 23% planning to use them soon. 74% of organizations use microservices for their web applications If you’re an IT leader, architect, or developer, you might have experienced the faster deployments, better fault isolation, and easier scaling that come with microservices. However, the old maxim, “ there’s no such thing as a free lunch ,” rings true here. Because microservices can also lead to significant operational overhead and complexity. Microservices sprawl, unclear component dependencies, service duplication, and anti-patterns like circular dependencies are all part of the chaos that comes hand-in-hand with microservices. there’s no such thing as a free lunch Perhaps you’ve seen some of the common symptoms of this microservices chaos: Engineering velocity drastically slows down Onboarding new developers grows more difficult Mean time to recovery (MTTR) for outages or performance issues increases Overall system resiliency diminishes, putting business outcomes at risk Engineering velocity drastically slows down Onboarding new developers grows more difficult Mean time to recovery (MTTR) for outages or performance issues increases Overall system resiliency diminishes, putting business outcomes at risk While it’s still best practice to modernize monolithic applications by breaking them into microservices, it’s also best practice to proactively guard against the issues that microservices introduce . it’s also best practice to proactively guard against the issues that microservices introduce Architecture Governance to Control the Chaos The best way to proactively guard against these problems is by using architecture governance — a collection of rules, processes, and practices that manage and control your software architecture. architecture governance With proper software architecture governance, you can reduce microservices complexity, ramp up developers faster, reduce MTTR, and improve the resiliency of your system, all while building a culture of intentionality. (Note that we’re talking about software architecture governance in this article, not enterprise architecture governance, which has been around for some time and is concerned with the policies, processes, and procedures of enterprise architecture strategies.) But exactly how do you build architecture governance? Governance, whether we’re talking about access, data, or architecture, is often just establishing a set of rules. But for architecture governance, you need more than just rules. You must start with architectural observability. Architectural Observability: The Foundation of Good Governance To implement effective governance, you first need comprehensive visibility into your software architecture . comprehensive visibility into your software architecture Architectural observability gives you a continuous, real-time view of your architecture. It provides your team with the necessary insights to manage the architecture and interconnectedness of distributed applications. Architectural observability Architectural observability provides documentation from a working system, it shows you systems flows, dependencies, what’s changing, and how your systems interact. It gives your team a thorough understanding of how the architecture works, how it drifts from release to release, and how its changes impact dependent services and resources. Without this visibility, you’re in trouble. If you’ve built a microservices project, you’ve probably encountered the results of a lack of visibility: Microservices that are difficult to manage Unclear component dependencies Service duplication Anti-patterns such as circular dependencies that increase the risk of outages Performance issues Microservices that are difficult to manage Unclear component dependencies Service duplication Anti-patterns such as circular dependencies that increase the risk of outages Performance issues Your overall application resiliency, scalability, and engineering velocity are at high risk. And it only gets worse — the longer you let this lack of visibility continue, the more complex and unmanageable your architecture becomes. Good Observability Equals Good Governance However, with good architectural observability, you can visualize your architecture layer in real-time, gain an understanding of how your system works, and monitor changes as your architecture evolves. with With observability, your team can proactively address issues, such as service sprawl and dependency mismanagement — before they lead to technical debt or system failures. And once you have that observability in place — once you know what’s happening and what’s changing — you can now create and enforce your rules of governance. You can implement a governance program that sits over your architecture, monitors, and controls changes, and allows you to build and manage microservices effectively by proactively applying standards and rules to your architecture and codebase as it evolves. Sounds great! But how do you actually build this? Let’s look at an example of how to build this in the real world. Implementing Architectural Observability and Governance First, you need to implement your architectural observability. The domain of architectural observability is new and rapidly evolving. For our example, we’ll use one of the platforms at the forefront — vFunction . vFunction is an AI-driven platform that provides visualizations of software architecture. vFunction vFunction leverages AI and OpenTelemetry tracing to enable teams to track every aspect of their architecture and monitor changes between releases. It can quickly identify complex flows, architectural, technical debt, and architectural drift. OpenTelemetry For example, vFunction allows you to better understand your distributed applications: Visualize the architecture and automatically create exportable sequence diagrams of all system and data flows. Identify drift, such as new services or dependencies. Identify resource exclusivity changes (such as multiple new services accessing the same database table) versus previous releases. Identify circular dependencies between services that could impact resiliency. Identify duplicate functionality/services that may be merged to remove complexity. Maintain an up-to-date, real-time view of the global microservices ecosystem. Visualize the architecture and automatically create exportable sequence diagrams of all system and data flows. Identify drift, such as new services or dependencies. Identify resource exclusivity changes (such as multiple new services accessing the same database table) versus previous releases. Identify circular dependencies between services that could impact resiliency. Identify duplicate functionality/services that may be merged to remove complexity. Maintain an up-to-date, real-time view of the global microservices ecosystem. vFunction uses the same data that APM tools use (so it’s straightforward to install) but uses a different layer of intelligence to address the architectural root cause of issues. This is especially important for managing the sprawl and complexity that result from the rapid and frequent deployment of microservices. Implementing Rules to Maintain Standards After implementing your observability, you’re ready to create and enforce your rules of governance. vFunction allows you to create architecture rules and tags and then monitor your architecture in real time against those rules, enforcing your critical governance policies. Any time a deployment or merge request breaks these rules, you can block that action. What type of rules might your governance enforce ? You could enforce rules such as: What type of rules might your governance enforce Dependency requirements. For example, certain services should or should not communicate with other services. Restricted resources. Multiple services should not depend on the same database table. Changes should not increase service interdependencies. Restrictions on new multi-hop flows that could impact performance and resiliency. Dependency requirements. For example, certain services should or should not communicate with other services. Restricted resources. Multiple services should not depend on the same database table. Changes should not increase service interdependencies. Restrictions on new multi-hop flows that could impact performance and resiliency. vFunction can watch and send alerts for all of the above. vFunction can watch and send alerts By implementing a focused set of rules and tags, you can create adaptive governance that guides developers in maintaining an optimal, clean, and efficient architecture. You can build architecture governance into your code and your culture. Conclusion Microservices chaos can threaten to derail development velocity, system resiliency, and operational efficiency. Architecture governance offers a way out of this chaos — a way to regain control over your architecture and steer it toward greater scalability and stability. Whether your organization is just starting to scale with microservices or is already in the depths of microservices sprawl, it’s the right time to introduce architecture governance to ensure that your architecture is efficient and resilient — and ready to support your next phase of growth.

Mastering Microservices: Building Effective Architecture Governance for Scalability and Stability

Authors:
(1) Chengrun Yang, Google DeepMind and Equal contribution;
(2) Xuezhi Wang, Google DeepMind;
(3) Yifeng Lu, Google DeepMind;
(4) Hanxiao Liu, Google DeepMind;
(5) Quoc V. Le, Google DeepMind;
(6) Denny Zhou, Google DeepMind;
(7) Xinyun Chen, Google DeepMind and Equal contribution. Table of Links Abstract and 1. Introduction 2 Opro: Llm as the Optimizer and 2.1 Desirables of Optimization by Llms 2.2 Meta-Prompt Design 3 Motivating Example: Mathematical Optimization and 3.1 Linear Regression 3.2 Traveling Salesman Problem (TSP) 4 Application: Prompt Optimization and 4.1 Problem Setup 4.2 Meta-Prompt Design 5 Prompt Optimization Experiments and 5.1 Evaluation Setup 5.2 Main Results 5.3 Ablation Studies 5.4 Overfitting Analysis in Prompt Optimization and 5.5 Comparison with Evoprompt 6 Related Work 7 Conclusion, Acknowledgments and References A Some Failure Cases B Prompting Formats for Scorer Llm C Meta-Prompts and C.1 Meta-Prompt for Math Optimization C.2 Meta-Prompt for Prompt Optimization D Prompt Optimization Curves on the Remaining Bbh Tasks E Prompt Optimization on Bbh Tasks – Tabulated Accuracies and Found Instructions B PROMPTING FORMATS FOR SCORER LLM Figure 14, 15, and 16 show examples of the Q_begin, Q_end, and A_begin prompting formats when the “QA” pattern is present. The “QA” pattern is eliminated when prompting instruction-tuned scorer models like text-bison with the Q_begin and Q_end formats (Figure 17 and 18). This paper is available on arxiv under CC0 1.0 DEED license. Authors: (1) Chengrun Yang, Google DeepMind and Equal contribution; (2) Xuezhi Wang, Google DeepMind; (3) Yifeng Lu, Google DeepMind; (4) Hanxiao Liu, Google DeepMind; (5) Quoc V. Le, Google DeepMind; (6) Denny Zhou, Google DeepMind; (7) Xinyun Chen, Google DeepMind and Equal contribution. Authors: Authors: (1) Chengrun Yang, Google DeepMind and Equal contribution; (2) Xuezhi Wang, Google DeepMind; (3) Yifeng Lu, Google DeepMind; (4) Hanxiao Liu, Google DeepMind; (5) Quoc V. Le, Google DeepMind; (6) Denny Zhou, Google DeepMind; (7) Xinyun Chen, Google DeepMind and Equal contribution. Table of Links Abstract and 1. Introduction Abstract and 1. Introduction 2 Opro: Llm as the Optimizer and 2.1 Desirables of Optimization by Llms 2 Opro: Llm as the Optimizer and 2.1 Desirables of Optimization by Llms 2.2 Meta-Prompt Design 2.2 Meta-Prompt Design 3 Motivating Example: Mathematical Optimization and 3.1 Linear Regression 3 Motivating Example: Mathematical Optimization and 3.1 Linear Regression 3.2 Traveling Salesman Problem (TSP) 3.2 Traveling Salesman Problem (TSP) 4 Application: Prompt Optimization and 4.1 Problem Setup 4 Application: Prompt Optimization and 4.1 Problem Setup 4.2 Meta-Prompt Design 4.2 Meta-Prompt Design 5 Prompt Optimization Experiments and 5.1 Evaluation Setup 5 Prompt Optimization Experiments and 5.1 Evaluation Setup 5.2 Main Results 5.2 Main Results 5.3 Ablation Studies 5.3 Ablation Studies 5.4 Overfitting Analysis in Prompt Optimization and 5.5 Comparison with Evoprompt 5.4 Overfitting Analysis in Prompt Optimization and 5.5 Comparison with Evoprompt 6 Related Work 6 Related Work 7 Conclusion, Acknowledgments and References 7 Conclusion, Acknowledgments and References A Some Failure Cases A Some Failure Cases B Prompting Formats for Scorer Llm B Prompting Formats for Scorer Llm C Meta-Prompts and C.1 Meta-Prompt for Math Optimization C Meta-Prompts and C.1 Meta-Prompt for Math Optimization C.2 Meta-Prompt for Prompt Optimization C.2 Meta-Prompt for Prompt Optimization D Prompt Optimization Curves on the Remaining Bbh Tasks D Prompt Optimization Curves on the Remaining Bbh Tasks E Prompt Optimization on Bbh Tasks – Tabulated Accuracies and Found Instructions E Prompt Optimization on Bbh Tasks – Tabulated Accuracies and Found Instructions B PROMPTING FORMATS FOR SCORER LLM Figure 14, 15, and 16 show examples of the Q_begin, Q_end, and A_begin prompting formats when the “QA” pattern is present. The “QA” pattern is eliminated when prompting instruction-tuned scorer models like text-bison with the Q_begin and Q_end formats (Figure 17 and 18). This paper is available on arxiv under CC0 1.0 DEED license. This paper is available on arxiv under CC0 1.0 DEED license. available on arxiv

Optimizing Scoring Models: Effective Prompting Formats

Authors:
(1) Chengrun Yang, Google DeepMind and Equal contribution;
(2) Xuezhi Wang, Google DeepMind;
(3) Yifeng Lu, Google DeepMind;
(4) Hanxiao Liu, Google DeepMind;
(5) Quoc V. Le, Google DeepMind;
(6) Denny Zhou, Google DeepMind;
(7) Xinyun Chen, Google DeepMind and Equal contribution. Table of Links Abstract and 1. Introduction 2 Opro: Llm as the Optimizer and 2.1 Desirables of Optimization by Llms 2.2 Meta-Prompt Design 3 Motivating Example: Mathematical Optimization and 3.1 Linear Regression 3.2 Traveling Salesman Problem (TSP) 4 Application: Prompt Optimization and 4.1 Problem Setup 4.2 Meta-Prompt Design 5 Prompt Optimization Experiments and 5.1 Evaluation Setup 5.2 Main Results 5.3 Ablation Studies 5.4 Overfitting Analysis in Prompt Optimization and 5.5 Comparison with Evoprompt 6 Related Work 7 Conclusion, Acknowledgments and References A Some Failure Cases B Prompting Formats for Scorer Llm C Meta-Prompts and C.1 Meta-Prompt for Math Optimization C.2 Meta-Prompt for Prompt Optimization D Prompt Optimization Curves on the Remaining Bbh Tasks E Prompt Optimization on Bbh Tasks – Tabulated Accuracies and Found Instructions 2.2 META-PROMPT DESIGN As the input to the LLM that acts as the optimizer, the meta-prompt contains the following two essential parts. Optimization problem description. The first part is the text description of the optimization problem, including the objective function and solution constraints. For example, for prompt optimization, the LLM can be instructed to “generate a new instruction that achieves a higher accuracy”, and we denote such instructions in the meta-prompt as meta-instructions. We can also provide customized meta-instructions as an informal regularization of the generated solutions, such as “the instruction should be concise and generally applicable”. Optimization trajectory. Besides understanding natural language instructions, LLMs are also shown to be able to recognize patterns from in-context demonstrations (Wei et al., 2023; Madaan & Yazdanbakhsh, 2022; Mirchandani et al., 2023). Our meta-prompt makes use of this property and instructs the LLM to leverage the optimization trajectory for generating new solutions. Specifically, the optimization trajectory includes past solutions paired with their optimization scores, sorted in the ascending order. Including optimization trajectory in the meta-prompt allows the LLM to identify similarities of solutions with high scores, encouraging the LLM to build upon existing good solutions to construct potentially better ones without the need of explicitly defining how the solution should be updated. 2.3 SOLUTION GENERATION At the solution generation step, the LLM generates new solutions with the meta-prompt as input. The following are the key optimization challenges we address in this stage. Optimization stability. In the optimization process, not all solutions achieve high scores and monotonically improve over prior ones. Due to the sensitivity of in-context learning to the prompt, LLM output can be drastically affected by low-quality solutions in the input optimization trajectory, especially at the beginning when the solution space has not been adequately explored. This sometimes results in optimization instability and large variance. To improve stability, we prompt the LLM to generate multiple solutions at each optimization step, allowing the LLM to simultaneously explore multiple possibilities and quickly discover promising directions to move forward. Exploration-exploitation trade-off. We tune the LLM sampling temperature to balance between exploration and exploitation. A lower temperature encourages the LLM to exploit the solution space around the previously found solutions and make small adaptations, while a high temperature allows the LLM to more aggressively explore solutions that can be notably different. This paper is available on arxiv under CC0 1.0 DEED license. Authors: (1) Chengrun Yang, Google DeepMind and Equal contribution; (2) Xuezhi Wang, Google DeepMind; (3) Yifeng Lu, Google DeepMind; (4) Hanxiao Liu, Google DeepMind; (5) Quoc V. Le, Google DeepMind; (6) Denny Zhou, Google DeepMind; (7) Xinyun Chen, Google DeepMind and Equal contribution. Authors: Authors: (1) Chengrun Yang, Google DeepMind and Equal contribution; (2) Xuezhi Wang, Google DeepMind; (3) Yifeng Lu, Google DeepMind; (4) Hanxiao Liu, Google DeepMind; (5) Quoc V. Le, Google DeepMind; (6) Denny Zhou, Google DeepMind; (7) Xinyun Chen, Google DeepMind and Equal contribution. Table of Links Abstract and 1. Introduction Abstract and 1. Introduction 2 Opro: Llm as the Optimizer and 2.1 Desirables of Optimization by Llms 2 Opro: Llm as the Optimizer and 2.1 Desirables of Optimization by Llms 2.2 Meta-Prompt Design 2.2 Meta-Prompt Design 3 Motivating Example: Mathematical Optimization and 3.1 Linear Regression 3 Motivating Example: Mathematical Optimization and 3.1 Linear Regression 3.2 Traveling Salesman Problem (TSP) 3.2 Traveling Salesman Problem (TSP) 4 Application: Prompt Optimization and 4.1 Problem Setup 4 Application: Prompt Optimization and 4.1 Problem Setup 4.2 Meta-Prompt Design 4.2 Meta-Prompt Design 5 Prompt Optimization Experiments and 5.1 Evaluation Setup 5 Prompt Optimization Experiments and 5.1 Evaluation Setup 5.2 Main Results 5.2 Main Results 5.3 Ablation Studies 5.3 Ablation Studies 5.4 Overfitting Analysis in Prompt Optimization and 5.5 Comparison with Evoprompt 5.4 Overfitting Analysis in Prompt Optimization and 5.5 Comparison with Evoprompt 6 Related Work 6 Related Work 7 Conclusion, Acknowledgments and References 7 Conclusion, Acknowledgments and References A Some Failure Cases A Some Failure Cases B Prompting Formats for Scorer Llm B Prompting Formats for Scorer Llm C Meta-Prompts and C.1 Meta-Prompt for Math Optimization C Meta-Prompts and C.1 Meta-Prompt for Math Optimization C.2 Meta-Prompt for Prompt Optimization C.2 Meta-Prompt for Prompt Optimization D Prompt Optimization Curves on the Remaining Bbh Tasks D Prompt Optimization Curves on the Remaining Bbh Tasks E Prompt Optimization on Bbh Tasks – Tabulated Accuracies and Found Instructions E Prompt Optimization on Bbh Tasks – Tabulated Accuracies and Found Instructions 2.2 META-PROMPT DESIGN As the input to the LLM that acts as the optimizer, the meta-prompt contains the following two essential parts. Optimization problem description. The first part is the text description of the optimization problem, including the objective function and solution constraints. For example, for prompt optimization, the LLM can be instructed to “generate a new instruction that achieves a higher accuracy”, and we denote such instructions in the meta-prompt as meta-instructions. We can also provide customized meta-instructions as an informal regularization of the generated solutions, such as “the instruction should be concise and generally applicable”. Optimization problem description. Optimization trajectory. Besides understanding natural language instructions, LLMs are also shown to be able to recognize patterns from in-context demonstrations (Wei et al., 2023; Madaan & Yazdanbakhsh, 2022; Mirchandani et al., 2023). Our meta-prompt makes use of this property and instructs the LLM to leverage the optimization trajectory for generating new solutions. Specifically, the optimization trajectory includes past solutions paired with their optimization scores, sorted in the ascending order. Including optimization trajectory in the meta-prompt allows the LLM to identify similarities of solutions with high scores, encouraging the LLM to build upon existing good solutions to construct potentially better ones without the need of explicitly defining how the solution should be updated. Optimization trajectory. 2.3 SOLUTION GENERATION At the solution generation step, the LLM generates new solutions with the meta-prompt as input. The following are the key optimization challenges we address in this stage. Optimization stability. In the optimization process, not all solutions achieve high scores and monotonically improve over prior ones. Due to the sensitivity of in-context learning to the prompt, LLM output can be drastically affected by low-quality solutions in the input optimization trajectory, especially at the beginning when the solution space has not been adequately explored. This sometimes results in optimization instability and large variance. To improve stability, we prompt the LLM to generate multiple solutions at each optimization step, allowing the LLM to simultaneously explore multiple possibilities and quickly discover promising directions to move forward. Optimization stability. Exploration-exploitation trade-off. We tune the LLM sampling temperature to balance between exploration and exploitation. A lower temperature encourages the LLM to exploit the solution space around the previously found solutions and make small adaptations, while a high temperature allows the LLM to more aggressively explore solutions that can be notably different. Exploration-exploitation trade-off. This paper is available on arxiv under CC0 1.0 DEED license. This paper is available on arxiv under CC0 1.0 DEED license. available on arxiv

Stories, News & Mentions about EFFECT

Effect’s Stories on HackerNoon

Effect's News & Mentions