Toloka, an Amsterdam-based <strong>Yandex</strong> subsidiary, is helping develop the facial recognition software Russia uses to massively track and arrest protesters, according to research by Follow the Money, The Bureau of Investigative Journalism, and Paper Trail Media. The two Russian companies Toloka works ...

Dutch Yandex subsidiary helping Russia with facial recognition software | NL Times

Cara nonton video bokeh di Yandex Browser Jepang. Akses ribuan video Indo Jepang dan Rusia secara gratis tanpa gangguan.

Tonton Video Bokeh Light Full HD Viral Indo di Yandex Browser Jepang Yandex RU dan Yandex EU Gratis Tanpa Sensor 2024 - poskota.co.id

Yandex Yalla Shoot TV Live Streaming Indonesia vs Libya Ilegal, Cek Link Resmi Prediksi Skor Susunan Pemain GRATIS NONTON INDOSIAR

Yandex Yalla Shoot TV Live Streaming Indonesia vs Libya Ilegal, Cek Link Resmi Prediksi Skor Susunan Pemain GRATIS NONTON INDOSIAR - Akurat Jateng

Yandex Live Streaming Timnas Indonesia vs Libya Ilegal, NONTON GRATIS INDOSIAR Cek Skor Babak Pertama LIVE SCORE Klik Link Resmi

Yandex Live Streaming Timnas Indonesia vs Libya Ilegal, NONTON GRATIS INDOSIAR Cek Skor Babak Pertama LIVE SCORE Klik Link Resmi - Akurat Jateng

Tribun-sulbar.com menyajikan berita dan video terkini seputar peristiwa, sepak bola, OTP37, selebriti, kesehatan, travel, hiburan dan wiki terkini dari Sulawesi Barat dan sekitarnya

Link Video Viral Terbaru 2024 Ramai di Yandex Com, Yandex ...

Latvia's electronic media regulator (NEPLP) announced Thursday that it has blocked access to all websites affiliated with Yandex.

Latvian government blocks all Yandex sites — Meduza

Компания «Яндекс» завершила перенос серверов yandex.kz в Казахстан, сообщила 26 декабря пресс-служба министерства цифрового развития инноваций и арокосмической ...

Власти Казахстана сообщили о завершении переноса серверов yandex.kz

Уже больше года Яндекс и Кремль ведут переговоры о попытке отделения российского бизнеса Яндекса от его голландской материнской компании Yandex NV.

Реструктуризацию Яндекса перенесли на 2024 год. Любая сделка потребует одобрения правительства России

The anticipated completion of Yandex's restructuring, a Nasdaq-listed Russian tech company is likely to be postponed until early 2024, Reuters reported. Often dubbed ‘Russia's Google’, Yandex has been in talks with the Kremlin for over a year to separate its Russian businesses from its ...

Yandex's restructuring faces delay, expected to extend into 2024 - Business & Economy News

MOSCOW (Reuters) - The completion of Nasdaq-listed Russian tech company Yandex (NASDAQ:YNDX)'s restructuring is expected to be delayed into next year, two sources familiar with the negotiations told Reuters on Monday.

Yandex's restructuring deal expected to be delayed to next year By Reuters

Yandex N.V.

yandex

Developing large language models requires substantial investments in time and GPU resources, translating directly into high costs. The larger the model, the more pronounced these challenges become. Recently, Yandex has introduced a new solution: YaFSDP, an open-source tool that promises to revolutionize LLM training by significantly reducing GPU resource consumption and training time. In a pre-training scenario involving a model with 70 billion parameters, using YaFSDP can save the resources of approximately 150 GPUs. This translates to potential monthly savings of roughly $0.5 to $1.5 million, depending on the virtual GPU provider or platform. Yandex has made YaFSDP publicly available on GitHub. The Challenge of Distributed LLM Training Training LLMs across multiple GPUs involves complex operations that lead to inefficiencies and high memory consumption. One of the main issues is the need to send and receive massive amounts of data between GPUs. For instance, in a typical all_reduce operation, twice the amount of gradient data as there are network parameters must be communicated. In the case of a Llama 70B model, this means transferring 280 GB of data per iteration. Furthermore, weights, gradients, and optimizer states are duplicated across GPUs, leading to an enormous memory load. The Llama 70B model and the Adam optimizer require over 1 TB of memory, far exceeding the typical 80 GB memory capacity of most GPUs. This redundancy severely slows down the training process and often makes it impractical to fit even moderately sized models into GPU memory. Introducing YaFSDP Yandex’s YaFSDP offers a highly effective solution to these challenges. By focusing on optimizing memory consumption and eliminating communication bottlenecks, YaFSDP enhances the efficiency of LLM training. It works by sharding layers instead of individual parameters, maintaining efficient communications, and avoiding redundant operations. Additionally, YaFSDP pre-allocates buffers for all required data, ensuring that the Torch allocator does not introduce inefficiencies. YaFSDP operates by utilizing two buffers for intermediate weights and gradients, with odd layers using one buffer and even layers using the other. The weights from different layers are stored in the same memory. If the layers have the same structure, they will always be identical. It is crucial to ensure that when you need layer X, the buffer contains the weights for layer X. All parameters will be stored in the corresponding memory chunk within the buffer. Memory Consumption During training, the primary memory consumers are weights, gradients, optimizer states, buffers, and activations. YaFSDP significantly reduces memory consumption by optimizing how these elements are stored and accessed. Weights, Gradients, and Optimizer States: These depend on the number of processes, and their memory consumption tends to approach zero as the number of processes increases. By sharding these components across GPUs, YaFSDP minimizes duplication and thus reduces memory usage. Buffers consume a constant amount of memory and store intermediate values during computations. Activations depend on the model size and the number of tokens processed per GPU. Activation Checkpointing Activation checkpointing is a technique that stores only necessary activations during the forward pass and recomputes them during the backward pass. This reduces the memory footprint significantly, as only essential data is stored. For example, in training a Llama 2 70B model with a batch size of 8192 tokens, activation storage can be reduced from over 110 GB to just 5 GB. However, this approach introduces additional computational overhead, which YaFSDP allows to avoid by not using the activation checkpointing for some layers which is possible due to memory optimization. Communication Optimization YaFSDP improves GPU communication efficiency by ensuring that data is transferred only when necessary and by overlapping communication with computation. It utilizes CUDA streams to manage concurrent computations and communications effectively. The tool uses two streams: a computation stream and a communication stream. Events synchronize these streams, ensuring that operations are executed in the correct order without introducing deadlocks. The forward pass on the third layer doesn’t start until the all_gather operation is completed (condition 1). Likewise, the all_gather operation on the third layer won’t begin until the forward pass on the first layer that uses the same buffer is completed (condition 2). Since there are no cycles in this scheme, deadlock is impossible. Experimental Results and Performance Gains The implementation of YaFSDP has shown remarkable improvements in training efficiency. In a pre-training scenario with a model having 70 billion parameters, YaFSDP was able to save the resources of approximately 150 GPUs. This translates into significant monthly cost savings, ranging from $0.5 to $1.5 million, depending on the virtual GPU provider or platform. YaFSDP reduces training time by up to 26% compared to existing methods like FSDP and optimizes memory usage, making it possible to train larger models more efficiently. Yandex has made YaFSDP publicly available on GitHub. ML engineers can leverage this tool to enhance the efficiency of their LLM training processes. By open-sourcing YaFSDP, Yandex aims to foster innovation and collaboration in the AI community, enabling developers to train models faster and cost-effectively. YaFSDP represents a significant advancement in LLM training. Addressing the critical challenges of memory consumption and communication inefficiencies enables faster and more efficient training of large language models. Developing large language models requires substantial investments in time and GPU resources, translating directly into high costs. The larger the model, the more pronounced these challenges become. Recently, Yandex has introduced a new solution: YaFSDP , an open-source tool that promises to revolutionize LLM training by significantly reducing GPU resource consumption and training time. In a pre-training scenario involving a model with 70 billion parameters, using YaFSDP can save the resources of approximately 150 GPUs. This translates to potential monthly savings of roughly $0.5 to $1.5 million, depending on the virtual GPU provider or platform. YaFSDP Yandex has made YaFSDP publicly available on GitHub . GitHub GitHub The Challenge of Distributed LLM Training The Challenge of Distributed LLM Training Training LLMs across multiple GPUs involves complex operations that lead to inefficiencies and high memory consumption. One of the main issues is the need to send and receive massive amounts of data between GPUs. For instance, in a typical all_reduce operation, twice the amount of gradient data as there are network parameters must be communicated. In the case of a Llama 70B model, this means transferring 280 GB of data per iteration. Furthermore, weights, gradients, and optimizer states are duplicated across GPUs, leading to an enormous memory load. The Llama 70B model and the Adam optimizer require over 1 TB of memory, far exceeding the typical 80 GB memory capacity of most GPUs. This redundancy severely slows down the training process and often makes it impractical to fit even moderately sized models into GPU memory. Introducing YaFSDP Introducing YaFSDP Yandex’s YaFSDP offers a highly effective solution to these challenges. By focusing on optimizing memory consumption and eliminating communication bottlenecks, YaFSDP enhances the efficiency of LLM training. It works by sharding layers instead of individual parameters, maintaining efficient communications, and avoiding redundant operations. Additionally, YaFSDP pre-allocates buffers for all required data, ensuring that the Torch allocator does not introduce inefficiencies. YaFSDP operates by utilizing two buffers for intermediate weights and gradients, with odd layers using one buffer and even layers using the other. The weights from different layers are stored in the same memory. If the layers have the same structure, they will always be identical. It is crucial to ensure that when you need layer X, the buffer contains the weights for layer X. All parameters will be stored in the corresponding memory chunk within the buffer. Memory Consumption Memory Consumption During training, the primary memory consumers are weights, gradients, optimizer states, buffers, and activations. YaFSDP significantly reduces memory consumption by optimizing how these elements are stored and accessed. Weights, Gradients, and Optimizer States: These depend on the number of processes, and their memory consumption tends to approach zero as the number of processes increases. By sharding these components across GPUs, YaFSDP minimizes duplication and thus reduces memory usage. Weights, Gradients, and Optimizer States : These depend on the number of processes, and their memory consumption tends to approach zero as the number of processes increases. By sharding these components across GPUs, YaFSDP minimizes duplication and thus reduces memory usage. Weights, Gradients, and Optimizer States Buffers consume a constant amount of memory and store intermediate values during computations. Buffers consume a constant amount of memory and store intermediate values during computations. Buffers Activations depend on the model size and the number of tokens processed per GPU. Activations depend on the model size and the number of tokens processed per GPU. Activations Activation Checkpointing Activation Checkpointing Activation checkpointing is a technique that stores only necessary activations during the forward pass and recomputes them during the backward pass. This reduces the memory footprint significantly, as only essential data is stored. For example, in training a Llama 2 70B model with a batch size of 8192 tokens, activation storage can be reduced from over 110 GB to just 5 GB. However, this approach introduces additional computational overhead, which YaFSDP allows to avoid by not using the activation checkpointing for some layers which is possible due to memory optimization. Communication Optimization Communication Optimization YaFSDP improves GPU communication efficiency by ensuring that data is transferred only when necessary and by overlapping communication with computation. It utilizes CUDA streams to manage concurrent computations and communications effectively. The tool uses two streams: a computation stream and a communication stream. Events synchronize these streams, ensuring that operations are executed in the correct order without introducing deadlocks. The forward pass on the third layer doesn’t start until the all_gather operation is completed (condition 1). Likewise, the all_gather operation on the third layer won’t begin until the forward pass on the first layer that uses the same buffer is completed (condition 2). Since there are no cycles in this scheme, deadlock is impossible. Experimental Results and Performance Gains Experimental Results and Performance Gains The implementation of YaFSDP has shown remarkable improvements in training efficiency. In a pre-training scenario with a model having 70 billion parameters, YaFSDP was able to save the resources of approximately 150 GPUs. This translates into significant monthly cost savings, ranging from $0.5 to $1.5 million, depending on the virtual GPU provider or platform. YaFSDP reduces training time by up to 26% compared to existing methods like FSDP and optimizes memory usage, making it possible to train larger models more efficiently. Yandex has made YaFSDP publicly available on GitHub . ML engineers can leverage this tool to enhance the efficiency of their LLM training processes. By open-sourcing YaFSDP, Yandex aims to foster innovation and collaboration in the AI community, enabling developers to train models faster and cost-effectively. GitHub GitHub YaFSDP represents a significant advancement in LLM training. Addressing the critical challenges of memory consumption and communication inefficiencies enables faster and more efficient training of large language models.

YaFSDP - An LLM Training Tool That Cuts GPU Usage by 20% - Is Out Now 

YaFSDP - একটি LLM প্রশিক্ষণ টুল যা GPU ব্যবহার 20% কম করে - এখন শেষ 

YaFSDP – ein LLM-Trainingstool, das die GPU-Nutzung um 20 % senkt – ist jetzt erhältlich

YaFSDP, una herramienta de formación LLM que reduce el uso de GPU en un 20 %, ya está disponible 

YaFSDP - Un outil de formation LLM qui réduit l'utilisation du GPU de 20 % - est maintenant disponible 

YaFSDP - एक LLM प्रशिक्षण उपकरण जो GPU के उपयोग को 20% तक कम करता है - अब उपलब्ध है

YaFSDP - GPU 使用率を 20% 削減する LLM トレーニング ツール - がリリースされました

YaFSDP - GPU 사용량을 20%까지 줄이는 LLM 교육 도구 - 출시됨 

YaFSDP - uma ferramenta de treinamento LLM que reduz o uso de GPU em 20% - já foi lançada 

YaFSDP — инструмент обучения LLM, который сокращает использование графического процессора на 20% — уже доступен 

YaFSDP - GPU Kullanımını %20 Oranında Azaltan Yüksek Lisans Eğitim Aracı - Artık Çıktı 

YaFSDP - Công cụ đào tạo LLM giúp cắt giảm 20% mức sử dụng GPU - Hiện đã ra mắt 

YaFSDP - 一款可将 GPU 使用率降低 20% 的 LLM 培训工具 - 现已推出

In the 1960s, the United States experienced a conglomerate boom. Companies such as Ling-Temco-Vought, Gulf and Western Industries, and others capitalized on low interest rates and periodically falling stock prices to acquire smaller companies through borrowed funds and often at temporarily reduced valuations. One distinctive feature of the conglomerate model was that the companies united within a conglomerate could operate in completely different and unrelated industries, ranging from meat packing to aerospace. The primary factors considered when deciding to purchase a new company for the conglomerate were exclusively financial indicators: the profitability and price of the acquired company. The main goal of this business model was to increase the conglomerate's stock value by growing earnings per share. Investors and the general public were also presented with more enticing ideas underlying the model, such as risk diversification of the conglomerate through the dissimilarity of its constituent businesses, as well as efficient access and redistribution of financial resources within the conglomerate. By 1968, the conglomerate business reached its peak, with approximately 4,500 companies being acquired in the US market, 26 of which made it into the top 500 American companies ranking. However, the stability of this model proved to be short-lived. In the late 1960s and early 1970s, conglomerate profits began to decline significantly due to rising interest rates. Alongside the decline in profits, market capitalization also fell, and conglomerates were forced to sell their assets to meet the debt obligations they had accumulated during the boom. If we look at the largest technology companies today (MAAMA), we can identify certain similarities with conglomerates. For instance, under the holding company Alphabet, created in 2015, Google encompasses a large number of companies from various business sectors. These include various internet services, self-driving cars, pharmaceutical and medical companies, and even agricultural technologies. Similarly, Amazon, in addition to its e-commerce business, includes Amazon Web Services — a cloud business, music and video streaming, the gaming streaming platform Twitch, as well as the natural products chain Whole Foods. Moreover, this phenomenon can be observed not only in the US but worldwide. For example, in China, Alibaba operates fintech, online travel, cloud, messaging, video streaming, gaming, and many other businesses alongside its main operations. And Russia's largest internet company, Yandex, is a leader not only in online search but also in taxi services and video streaming, and it is developing several other areas. At first glance, it may seem that conglomerates have risen from the ashes like a phoenix in modern technology giants that attempt to engage in everything at once. But is that really the case? Are modern technology corporations truly a new wave of conglomerate models? And are they destined for the inevitable collapse that occurred with conglomerates in the 1970s? Let's take a closer look at the businesses united under the brands of technology giants. Through analyzing their press releases, media statements, and strategic materials, it becomes evident that the development of specific services is primarily driven by strategic considerations rather than purely financial motives. The companies within these groups are closely integrated with each other, working towards the achievement of specific strategic goals, even if these connections may not be immediately apparent. For instance, Amazon's video streaming service, Amazon Prime Video, doesn't exist in a separate universe from its e-commerce business but functions as part of the broader Amazon Prime subscription, which includes free shipping and other services. Another example is AWS, which originated from Amazon's endeavour to leverage its expertise in building efficient data centres for its own needs. Facebook's acquisition of Oculus similarly goes beyond mere profitability from selling VR/AR devices; it aims to integrate social networks with this technology to create a so-called metaverse. Thus, the fundamental difference between modern technology corporations and conglomerates lies in the existence of such synergistic relationships between the assets under their management. In contrast to traditional conglomerates that primarily focused on increasing profits through financial leverage, technology companies actively interconnect their businesses to mutually enhance their products and services, delivering an exceptional customer experience. This paradigm shift has given rise to the concept of a 'technology ecosystem' or 'digital ecosystem,' capturing the interconnectedness and collaborative nature of these integrated models. There can be endless examples of such integration, but the most common ones include: Cross-promotion of one product on the platforms of another (e.g., installing Apple Music on all Apple devices or promoting Google services through its search engine). Shared logins across different services to ensure a seamless user experience and efficient data exchange between services (Apple ID, Google Account). Subscription bundles that allow users to subscribe to a set of ecosystem services simultaneously (Amazon Prime, Yandex Plus). Loyalty programs that enable users to accumulate and redeem cash back points within the ecosystem. Common umbrella brand name. Utilization of shared infrastructure and architecture to save costs. And more. By establishing such close relationships between products within the ecosystem, they can act as instruments in an orchestra, playing different parts on the path to achieving the strategic goals of the entire group. For instance, Amazon Video doesn't necessarily have to be profitable on its own as long as it brings new users and keeps them engaged on the e-commerce platform. It is impossible to imagine such a case outside the ecosystem model. This fact significantly transforms the product landscape and offers users services that would never have existed without the ecosystem model. Products within ecosystems, based on their functional purpose (although sometimes one product can serve multiple functions simultaneously), can be divided into four groups: Customer Acquisition Products whose primary goal is to attract new users to the ecosystem. Examples of such products could be online retail for Amazon or a search portal for Google. Typically, these are core products of the ecosystem, with the highest visibility and user benefits. Customer Retention Products with a high frequency of use and a high cost of switching to a competing service for users. Thanks to decision inertia, these products help retain customers within the ecosystem. Examples include loyalty programs within the ecosystem that create the illusion for users that it is better to stay within the ecosystem and spend accumulated points rather than switch to a more advantageous competitor's product. Another example is Amazon Video, whose unique content is not available on other video platforms. Consequently, a user who has already started watching a series will want to finish it, thereby continuing to pay for the Amazon Prime subscription. And with the added benefit of free shipping included in Amazon Prime, why not make another purchase or two on the Amazon marketplace? Monetization The most obvious functional load. After attracting and retaining users within the ecosystem, it is necessary to monetize them. A prime example of products in this category can be highly-margin goods that users would never purchase without being part of the ecosystem (think about Apple's chargers and cables and their prices). Moon Shots Products that do not fit into any of the three aforementioned categories but have the potential to create new growth opportunities or prevent disruption in the company's industry. Examples include voice assistants, developments in autonomous technologies, and more. How successful can this model be considering the drawbacks inherently associated with conglomerates and similar structures? Specifically, the need to create an additional layer of bureaucracy to manage a large group of businesses, management defocus, and the development of a large number of culturally different businesses within the same corporate standards of the ecosystem. Answering this question based on data and numerical metrics is quite challenging. Firstly, it is very difficult for companies to accurately measure and quantify the positive effects of the synergies between products within their ecosystems described above. Secondly, companies often do not disclose detailed information regarding such effects in their financial reports or public statements. Taking the example of Amazon Video, various assessments and comments indicate that Amazon Prime users consider the presence of the video streaming service as the second reason for subscribing, with only ‘free 1-day shipping’ ranking higher. It is reported that Prime Video generates approximately 20-30% of new trial users for the subscription. Furthermore, compared to regular users, Amazon Prime users make purchases on the marketplace 2.3 times more frequently and spend 2.5 times more. However, assessing the extent to which these positive effects compensate for Amazon's massive multi-billion-dollar expenses in creating new content remains a complex challenge. Assessing the success of the ecosystem model from a different perspective, we can examine the ranking of the world's largest technology companies by market capitalization. Notably, the top four positions are predominantly occupied by ecosystem-driven companies (Apple, Amazon, Microsoft, and Alphabet), while monoline companies like Nvidia and Tesla appear further down the list at the fifth and sixth spots, respectively. Even Netflix, a company focused on a single business model, has recently dropped out of the acronym for the most prominent technology companies in the world, FAANG, and now faces significant pressure from ecosystem players like Amazon Video. In conclusion, modern technology giants demonstrate distinct differences from the conglomerates of the past. Their approach to launching or acquiring new businesses is not driven solely by financial engineering but by the intention to create a synergistic set of products that complement each other and deliver enhanced value collectively. Within these ecosystems, products may serve different functional purposes and prioritize factors beyond immediate profit extraction. This transformative concept reshapes the business landscape, offering users an array of products and services that would not exist without the presence of these ecosystems. While it is still too early to definitively determine the long-term success of ecosystems from an economic history perspective, indirect indicators suggest the viability and advantages of this business model compared to companies focused solely on a single model of development.

Tech Ecosystems: Reshaping Business Landscape — A Revolution Beyond Conglomerates

Hi there! This is a continuation of my for stress testing your services. The documentation mentions that the main engine for load is Phantom, but it can be replaced with Pandora if you need to create unique test scenarios or work with other types of requests (such as grpc or any other binaries). In this article, we will explore how to integrate Yandex Pandora for two cases: first article on how to use Yandex Tank GRPC flatbuffer (over http) As I mentioned before, our main goal is to use a simple tool with straightforward documentation that doesn’t require hiring a bunch of people to set up the test. Pandora, like Tank, has very simple and understandable documentation, does exactly what we need, and has a low entry threshold. In my case, the primary stack is Golang, and I choose solutions around it. https://hackernoon.com/turbocharge-load-testing-yandextank-ghz-combo-for-lightning-fast-code-checks?embedable=true Let’s start by referring to the documentation and see what Pandora is all about. “Pandora is a high-performance load generator in the Go language. It has built-in HTTP(S) and HTTP/2 support, and you can write your own load scenarios in Go, compiling them just before your test.” Since we are interested in scenarios, let’s take a look at the documentation: https://yandex.github.io/pandora/eng/custom.html?source=post_page-----12c250f2bff2--------------------------------&embedable=true As we can see from the documentation, there is an excellent example of how to write our “gun” to create our load engine. Great, let’s take this example and copy-paste it into our repository. Next, let’s look at our service contracts: syntax = "proto3";

option go_package = "/docs";

service DocumentService {
 rpc GetAllByLimitAndOffset(GetAllRequest) returns (GetAllResponse) {}
 rpc Save(SaveRequest) returns (SaveResponse) {}
 rpc Validate(ValidateRequest) returns (ValidateResponse) {}
} Our task involves covering two methods ( ): yes, we’ll skip save method for current case Save Validate Create a Yandex-tank folder, create a GRPC folder inside it, move our PROTO file there, and generate contracts with the command: protoc - go_out=./docs - go_opt=paths=source_relative \
 - go-grpc_out=./docs - go-grpc_opt=paths=source_relative docs.proto After that, we will get files as shown in the screenshot. And now, let’s start to write our own gun. We have to add a huge number of imports and create a struct for Custom params: package main

import (
 "context"
 "log"
 "strconv"
 "time"

 _ "github.com/golang/protobuf/ptypes/timestamp"
 "github.com/spf13/afero"
 "google.golang.org/grpc"
 pb "grpc-gun/docs"

 "github.com/yandex/pandora/cli"
 "github.com/yandex/pandora/components/phttp/import"
 "github.com/yandex/pandora/core"
 "github.com/yandex/pandora/core/aggregator/netsample"
 "github.com/yandex/pandora/core/import"
 "github.com/yandex/pandora/core/register"
)

type Ammo struct {
 Tag    string
 Param1 string
 Param2 string
 Param3 string
} Now, we should add and where we can add our grpc client (just copy and paste from official docs) GunConfig Gun, type GunConfig struct {
 Target string `validate:"required"` // Configuration will fail, without target defined
}

type Gun struct {
 // Configured on construction.
 client grpc.ClientConn
 conf   GunConfig
 // Configured on Bind, before shooting
 aggr core.Aggregator // Maybe your custom Aggregator.
 core.GunDeps
} And add new func for Gun, almost done! func NewGun(conf GunConfig) *Gun {
   return &Gun{conf: conf}
} We can see, that we should add a little bit more for running: – Bind method. We just bind our client and args: – override Shoot method func (g *Gun) Bind(aggr core.Aggregator, deps core.GunDeps) error {
 // create gRPC stub at gun initialization
 conn, err := grpc.Dial(
  g.conf.Target,
  grpc.WithInsecure(),
  grpc.WithTimeout(time.Second),
  grpc.WithUserAgent("load test, pandora custom shooter"))
 if err != nil {
  log.Fatalf("FATAL: %s", err)
 }
 g.client = *conn
 g.aggr = aggr
 g.GunDeps = deps
 return nil
}

func (g *Gun) Shoot(ammo core.Ammo) {
 customAmmo := ammo.(*Ammo)
 g.shoot(customAmmo)
} It’s time to add switch method for SaveCase and ValidateCase requests and to add methods, as example for ValidateCase: func (g *Gun) caseValidate(client pb.DocumentServiceClient, ammo *Ammo) int {
 code := 0
 docName := ammo.Param1

 out, err := client.Validate(
  context.TODO(), &pb.ValidateRequest{Document: createDoc(docName)},
 )

 if err != nil {
  log.Printf("FATAL: %s", err)
  code = 500
 }

 if out != nil {
  code = 200
 }
 return code
} Full code here: https://github.com/IliaEre/serialisation-contest/tree/main/common/loadtest/yandex-tank/grpc?source=post_page-----12c250f2bff2--------------------------------&embedable=true Alright, pay close attention! func main() {
     // Standard imports.
     fs := afero.NewOsFs()
     coreimport.Import(fs)
     // May not be imported, if you don't need http guns, etc.
     phttp.Import(fs)
    // Custom imports. Integrate your custom types into configuration system.
     coreimport.RegisterCustomJSONProvider("grpc_provider", func() core.Ammo { return &Ammo{} })
    register.Gun("grpc_gun", NewGun, func() GunConfig {
     return GunConfig{
     Target: "localhost:84",
     }
     })
    cli.Run()
} When creating our gun, we must specify everything to compile and run it. We create a provider, specifying the name “grpc_provider” We create a gun and specify the name “grpc_gun” In the target, we specify**“localhost:84”** the location where our service is accessible. Okay, let’s continue. We have successfully copied and adjusted the code for the gun. Now, how do we run it, and what do we need? First, let’s create ammo again. This time, we’ll simply describe them like this: {"tag": "/ValidateCase", "Param1": "validate_doc_grpc_pandora"} This is necessary so that, in the method, we can retrieve the parameters needed for the test. In our case, we will retrieve the document name specified in Param1. Our code will automatically pass this parameter to the ValidateCase method, and we’ll be able to retrieve it. Great, the ammo is ready. Now, let’s configure the test itself: pools:
  - id: HTTP pool
    gun:
      type: grpc_gun   # custom gun name (specified at `register.Gun("my_custom_gun_name", ...`)
      target: "localhost:84"
    ammo:
      type: grpc_provider
      source:
        type: file
        path: validate-json.ammo
    result:
      type: phout
      destination: ./phout.log
    rps: { duration: 60s, type: const, ops: 1000 }
    startup:
      type: once
      times: 10 Here, we specify our ammo as “validate-json.ammo” Their type is “grpc_provider” Our gun is “grpc_gun” The load is 1000 rps for 60 seconds: rps: { duration: 60s, type: const, ops: 1000 } And the output goes to the file “./phout.log” — Yandex Tank understands this. The finishing straight — let’s now perform three simple actions: Build our gun. Assemble a new configurator for the tank. Run tests and check the results. Building it is very straightforward — just like a simple Go application: GOOS=linux GOARCH=amd64 go build It’s important to specify `GOOS=linux GOARCH=amd64`; otherwise, you might encounter errors when launching the tank. That’s it; the binary file is ready. Let’s update another configurator: phantom:
  enabled: false
pandora:
  enabled: true
  package: yandextank.plugins.Pandora
  pandora_cmd: ./grpc-gun # Pandora executable path
  config_file: ./validate-load.yml # Pandora config path
overload:
  enabled: true
  package: yandextank.plugins.DataUploader
  job_name: "grpc validate report"
  token_file: "env/token.txt"
telegraf:
  enabled: false
autostop:
  autostop:
    - time(1s,10s) # if request average > 1s
    - http(5xx,100%,1s) # if 500 errors > 1s
    - http(4xx,25%,10s) # if 400 > 25%
    - net(xx,25,10) # if amount of non-zero net-codes in every second of last 10s period is more than 25 Specify where our gun and its settings are located. Leave everything else as before and run it all with this script: echo "run tank"
docker run \
 -v $(pwd):/var/loadtest \
 - net="host" \
 -it direvius/yandex-tank -c validate-tank-load.yml After that, we will see our results. Special gun for flatbuffer! For flatbuffer, the story repeats. All you need to do is adjust your client code from grpc to http. Here’s an example: type Gun struct {
   // Configured on construction.
   client http.Client
   conf GunConfig
   // Configured on Bind, before shooting.
   aggr core.Aggregator // Maybe your custom Aggregator.
   core.GunDeps
   files [][]byte
}

func NewGun(conf GunConfig) *Gun {
   return &Gun{conf: conf}
}

func (g *Gun) Bind(aggr core.Aggregator, deps core.GunDeps) error {
   c := http.Client{Timeout: time.Duration(1) * time.Second}
   g.client = c
   g.GunDeps = deps
   g.aggr = aggr
} And then create similar methods and describe their behavior: func (g *Gun) caseValidate(client *http.Client) int {
  host := g.conf.Target
  response, err := client.Post(host+"/report/validate", "application/octet-stream", bytes.NewBuffer(g.files[2]))
  if err != nil {
   log.Printf("FATAL: %s", err)
   return 500
  }
  return response.StatusCode
} That’s it! In conclusion We’ve explored a new tool for performance testing called Yandex Pandora. You can run it as a standalone load engine if you don’t need fancy metrics. Alternatively, you can run it based on Yandex Tank, and in this case, you’ll get beautiful graphs and all the features of this cool tool! That’s all, I hope it was helpful! Thank you. Full code here: https://github.com/IliaEre/serialisation-contest/tree/main/common/loadtest/yandex-tank?embedable=true Also published . here

Leveraging Yandex Pandora: Stress Testing GRPC and Flatbuffer Services

Hi, there! Occasionally, there arises a need for swift load testing, whether it be in a local environment or on a testing platform. Typically, such tasks are tackled using specialized tools that demand thorough prior comprehension. However, within enterprises and startups where rapid time-to-market and prompt hypothesis validation are paramount, excessive tool familiarization becomes a luxury. This article aims to spotlight developer-centric solutions that obviate the necessity for profound engagement, allowing for rudimentary testing without delving into pages of documentation. local running You should install:: Docker — all services and tools are required for it. Java 19+ — for kotlin service. Also, you can try to use the Java 8 version; it should work, but you have to change the Gradle settings. Golang — one of the services is golang service =) Python 3+ — for the Yandex tank. Tech Requirements Prior to embarking on our journey, it is advisable to generate a couple of services that can serve as illustrative examples for testing purposes. Kotlin + webflux. r2dbc + postgres. Stack: Our service has: – get all stocks (limit 10) GET – get stock by name GET__ /api/v1/stocks /api/v1/stock ?name=appl – save stock POST / api/v1/stock It should be an easy service because we have to focus on load testing =) Kotlin and the HTTP Service Let's start by creating a small service with some basic logic inside. We'll prepare a model for this purpose: @Table("stocks")
data class Stock(
    @field:Id val id: Long?,
    val name: String,
    val price: BigDecimal,
    val description: String
) Simple router: @Configuration
@EnableConfigurationProperties(ServerProperties::class)
class StockRouter(
    private val properties: ServerProperties,
    private val stockHandler: StockHandler
) {

    @Bean
    fun router() = coRouter {
        with(properties) {
            main.nest {
                contentType(APPLICATION_JSON).nest {
                    POST(save, stockHandler::save)
                }
                GET(find, stockHandler::find)
                GET(findAll, stockHandler::findAll)
            }
        }
    }
} and handler: @Service
class StockHandlerImpl(
    private val stockService: StockService
) : StockHandler {

    private val logger = KotlinLogging.logger {}

    private companion object {
        const val DEFAULT_SIZE = 10
        const val NAME_PARAM = "name"
    }

    override suspend fun findAll(req: ServerRequest): ServerResponse {
        logger.debug { "Processing find all request: $req" }

        val stocks = stockService.getAll(DEFAULT_SIZE)
        return ServerResponse.ok()
            .contentType(MediaType.APPLICATION_JSON)
            .body(stocks, StockDto::class.java)
            .awaitSingle()
    }

    override suspend fun find(req: ServerRequest): ServerResponse {
        logger.debug { "Processing find all request: $req" }

        val name = req.queryParam(NAME_PARAM)

        return if (name.isEmpty) {
            ServerResponse.badRequest().buildAndAwait()
        } else {
            val stocks = stockService.find(name.get())

            ServerResponse.ok()
                .contentType(MediaType.APPLICATION_JSON)
                .body(stocks, StockDto::class.java)
                .awaitSingle()

        }
    }

    override suspend fun save(req: ServerRequest): ServerResponse {
        logger.debug { "Processing save request: $req" }

        val stockDto = req.awaitBodyOrNull(StockDto::class)
        return stockDto?.let { dto ->
            stockService.save(dto)

            ServerResponse
                .ok()
                .contentType(MediaType.APPLICATION_JSON)
                .bodyValue(dto)
                .awaitSingle()
        } ?: ServerResponse.badRequest().buildAndAwait()
    }

} Full code here: GitHub Create a docker file: FROM openjdk:20-jdk-slim
VOLUME /tmp
COPY build/libs/*.jar app.jar
ENTRYPOINT ["java", "-Dspring.profiles.active=stg", "-jar", "/app.jar"] Then, build a docker image and tune it 🤤 docker build -t ere/stock-service .
docker run -p 8085:8085 ere/stock-service But for now, it's better to stick with the idea of running everything through Docker containers and migrate our service into a Docker Compose setup. version: '3.1'

services:

  db:
    image: postgres
    container_name: postgres-stocks
    ports:
      - "5432:5432"
    environment:
      POSTGRES_PASSWORD: postgres

  adminer:
    image: adminer
    ports:
      - "8080:8080"

  stock-service:
    image: ere/stock-service
    container_name: stock-service
    ports:
      - "8085:8085"
    depends_on:
      - db Moving forward: how can we proceed with testing? Specifically, how can we initiate a modest load test for our recently developed service? It’s imperative that the testing solution is both straightforward to install and user-friendly. Given our time constraints, delving into extensive documentation and articles isn’t a viable option. Fortunately, there’s a viable alternative — enter Yandex Tank. The tank is a powerful instrument for testing and has important integrations with , but in the article, we will use it as a simple tool. JMeter https://github.com/yandex/yandex-tank?embedable=true Docs: https://yandextank.readthedocs.org/en/latest/ Let's kick off by creating a folder for our tests. Once we've placed the configs and other essential files—fortunately, just a couple of them—we'll be all set. For our service, we need to test methods “get-all” and “save.” The first config for find method. phantom:
  address: localhost
  port: "8085"
  load_profile:
    load_type: rps
    schedule: line(100, 250, 30s)
  writelog: all
  ssl: false
  connection_test: true
  uris:
    - /api/v1/stocks
overload:
  enabled: false
telegraf:
  enabled: false  
autostop:
  autostop:
    - time(1s,10s) # if request average > 1s
    - http(5xx,100%,1s) # if 500 errors > 1s
    - http(4xx,25%,10s) # if 400 > 25%
    - net(xx,25,10) # if amount of non-zero net-codes in every second of last 10s period is more than 25 Key settings for configuration: Address and port: Same as our application. Load test profile (load_profile): We'll use the 'lined' type, ranging from 100 requests per second to 250 with a 30-second limit. URIs: A list of URLs to be tested. Autostop pattern: No need to stress-test if our service has already gone down! =) Copy and paste the bash script (tank sh): docker run \
    -v $(pwd):/var/loadtest \
    --net="host" \
    -it yandex/yandex-tank And run! What will we see as a result? Yandex Tank will log everything it deems worthy during the test. We can observe metrics such as the 99th percentile and requests per second (rps). So, are we stuck with the terminal now? I want a GUI! Don't worry, Yandex Tank has a solution for that too. We can utilize one of the overload plugins. Here's an example of how to add it: overload:
  enabled: true
  package: yandextank.plugins.DataUploader
  job_name: "save docs"
  token_file: "env/token.txt" We should add our token; just go here and logic by GitHub: https://overload.yandex.net Okay, dealing with a GET request is straightforward, but what about POST? How do we structure the request? The thing is, you can't just throw the request into the tank; you need to create patterns for it! What are these patterns? It's simple — you need to write a small script, which you can again fetch from the documentation and tweak a bit to suit our needs. https://yandextank.readthedocs.io/en/latest/ammo_generators.html?source=post_page-----77157bf4a779--------------------------------&embedable=true And we should add our own body and headers: #!/usr/bin/env python3
# -*- coding: utf-8 -*-

import sys
import json

# http request with entity body template
req_template_w_entity_body = (
        "%s %s HTTP/1.1\r\n"
        "%s\r\n"
        "Content-Length: %d\r\n"
        "\r\n"
        "%s\r\n"
    )

# phantom ammo template
ammo_template = (
        "%d %s\n"
        "%s"
    )

method = "POST"
case = ""
headers = "Host: test.com\r\n" + \
          "User-Agent: tank\r\n" + \
          "Accept: */*\r\n" + \
          "Connection: Close\r\n" 


def make_ammo(method, url, headers, case, body):
    """ makes phantom ammo """
    req = req_template_w_entity_body % (method, url, headers, len(body), body)
    return ammo_template % (len(req), case, req)


def generate_json():
    body = {
        "name": "content",
        "price": 1,
        "description": "description"
    }
    url = "/api/v1/stock"
    h = headers + "Content-type: application/json"
    s1 = json.dumps(body)
    ammo = make_ammo(method, url, h, case, s1)
    sys.stdout.write(ammo)
    f2 = open("ammo/ammo-json.txt", "w")
    f2.write(ammo)


if __name__ == "__main__":
    generate_json() Result: 212 
POST /api/v1/stock HTTP/1.1
Host: test.com
User-Agent: tank
Accept: */*
Connection: Close
Content-type: application/json
Content-Length: 61

{"name": "content", "price": 1, "description": "description"} That’s it! Just run the script, and we will have ammo-json.txt. Just set new params to config, and delete the URLs: phantom:
  address: localhost:9001
  ammo_type: phantom
  ammofile: ammo-json.txt And run it one more time! It’s Time to Test the GRPC! Having acquainted ourselves with loading HTTP methods, it’s natural to consider the scenario for GRPC. Are we fortunate enough to have an equally accessible tool for GRPC, akin to the simplicity of a tank? The answer is affirmative. Allow me to introduce you to ‘ghz’. Just take a look: https://ghz.sh/?source=post_page-----77157bf4a779--------------------------------&embedable=true But before we do that, we should create a small service with Go and GRPC as a good test service. Prepare a small proto file: syntax = "proto3";

option go_package = "stock-grpc-service/stocks";

package stocks;

service StocksService {
  rpc Save(SaveRequest) returns (SaveResponse) {}
  rpc Find(FindRequest) returns (FindResponse) {}
}

message SaveRequest {
  Stock stock = 1;
}

message SaveResponse {
  string code = 1;
}

message Stock {
  string name = 1;
  float price = 2;
  string description = 3;
}

message FindRequest {
  enum Type {
    INVALID = 0;
    BY_NAME = 1;
  }
  message ByName {
    string name = 1;
  }
  Type type = 1;

  oneof body {
    ByName by_name = 2;
  }
}

message FindResponse {
  Stock stock = 1;
} And generate it! (also, we should install ) protoc protoc --go_out=. --go_opt=paths=source_relative --go-grpc_out=. --go-grpc_opt=paths=source_relative stocks.proto Our results: Coding Time! Next steps: Create services as fast as we can. Create dto (stock entity for db layer) package models

// Stock – base dto
type Stock struct {
 ID          *int64  `json:"Id"`
 Price       float32 `json:"Price"`
 Name        string  `json:"Name"`
 Description string  `json:"Description"`
} Implement server // Server is used to implement stocks.UnimplementedStocksServiceServer.
type Server struct {
 pb.UnimplementedStocksServiceServer
 stockUC stock.UseCase
}

// NewStockGRPCService stock gRPC service constructor
func NewStockGRPCService(emailUC stock.UseCase) *Server {
 return &Server{stockUC: emailUC}
}

func (e *Server) Save(ctx context.Context, request *stocks.SaveRequest) (*stocks.SaveResponse, error) {
 model := request.Stock

 stockDto := &models.Stock{
  ID:          nil,
  Price:       model.Price,
  Name:        model.Name,
  Description: model.Description,
 }

 err := e.stockUC.Create(ctx, stockDto)
 if err != nil {
  return nil, err
 }

 return &stocks.SaveResponse{Code: "ok"}, nil
}

func (e *Server) Find(ctx context.Context, request *stocks.FindRequest) (*stocks.FindResponse, error) {
 code := request.GetByName().GetName()

 model, err := e.stockUC.GetByID(ctx, code)
 if err != nil {
  return nil, err
 }

 response := &stocks.FindResponse{Stock: &stocks.Stock{
  Name:        model.Name,
  Price:       model.Price,
  Description: model.Description,
 }}

 return response, nil
} Full code here: click, please! Test It! Install GHz with brew (as usual): link Let’s check a simple example: link Now, we should change it a little bit: move to the folder with the proto files. add method: . stocks.StocksService.Save add simple body: ‘{“stock”: { “name”:”APPL”, “price”: “1.3”, “description”: “apple stocks”} }’. connections will be shared among goroutine workers. Each pair of goroutines will share a single connection. 10 20 2 set service’s port and the result: cd .. && cd stock-grpc-service/proto

ghz --insecure \
  --proto ./stocks.proto \
  --call stocks.StocksService.Save \
  -d '{"stock": { "name":"APPL", "price": "1.3", "description": "apple stocks"} }' \
  -n 2000 \
  -c 20 \
  --connections=10 \
  0.0.0.0:5007 Run it! Summary:
  Count:        2000
  Total:        995.93 ms
  Slowest:      30.27 ms
  Fastest:      3.11 ms
  Average:      9.19 ms
  Requests/sec: 2008.16

Response time histogram:
  3.111  [1]   |
  5.827  [229] |∎∎∎∎∎∎∎∎∎∎∎
  8.542  [840] |∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎
  11.258 [548] |∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎∎
  13.973 [190] |∎∎∎∎∎∎∎∎∎
  16.689 [93]  |∎∎∎∎
  19.405 [33]  |∎∎
  22.120 [29]  |∎
  24.836 [26]  |∎
  27.551 [6]   |
  30.267 [5]   |

Latency distribution:
  10 % in 5.68 ms 
  25 % in 6.67 ms 
  50 % in 8.27 ms 
  75 % in 10.49 ms 
  90 % in 13.88 ms 
  95 % in 16.64 ms 
  99 % in 24.54 ms 

Status code distribution:
  [OK]   2000 responses And what, stare at everything in the terminal again? No, with ghz, you can also generate a report, but unlike Yandex, it will be generated locally and can be opened in the browser. Just set it: ghz --insecure -O html -o reports_find.html \
... -O + html → output format -o filename Conclusion In summary, when you need a swift assessment of your service's ability to handle a load of 100+ requests per second or identify potential weaknesses, there's no need to initiate intricate processes involving teams, seeking assistance from AQA, or relying on the infrastructure team. More often than not, developers have capable laptops and computers that can execute a small load test. So, go ahead and give it a shot—save yourself some time! I trust you found this brief article beneficial. Valuable Documentation I Recommend Reading: Yandex Tank: docs link Yandex Tank GitHub: GitHub link Yandex Tank Setting: link ghz official page: link ghz setting: ghz config: link link May the Force Be With You! Thanks once again, and best of luck! 🍀🕵🏻

Turbocharge Load Testing: YandexTank + ghz Combo for Lightning–Fast Code Checks!

টার্বোচার্জ লোড টেস্টিং: লাইটনিং-ফাস্ট কোড চেকের জন্য ইয়ানডেক্সট্যাঙ্ক + গিগাহার্টজ কম্বো!

Turbolader-Lasttest: YandexTank + GHz-Kombination für blitzschnelle Codeprüfungen!

Prueba de carga de turbocompresor: combinación YandexTank + ghz para verificaciones de códigos ultrarrápidas.

Test de charge Turbocharge : YandexTank + ghz Combo pour des vérifications de code ultra-rapides !

टर्बोचार्ज लोड परीक्षण: लाइटनिंग-फास्ट कोड जांच के लिए YandexTank + ghz कॉम्बो!

ターボチャージ負荷テスト: YandexTank + ghz コンボによる超高速コード チェック!

터보차지 부하 테스트: 매우 빠른 코드 검사를 위한 YandexTank + ghz 콤보!

Teste de carga turboalimentada: combinação YandexTank + ghz para verificações de código extremamente rápidas!

Турбо-нагрузочное тестирование: комбинация ЯндексТанк + ghz для молниеносной проверки кода!

Turboşarj Yük Testi: Hızlı Kod Kontrolleri için YandexTank + ghz Kombinasyonu!

Kiểm tra tải tăng áp: Combo YandexTank + ghz để kiểm tra mã nhanh như chớp!

涡轮增压负载测试：YandexTank + ghz 组合可实现闪电般的快速代码检查！

Below we will describe in detail how to set up smart banners to provide the highest conversion rates. If you run ads in Yandex.Direct, do not limit yourself to standard campaigns in YAN and search engines. Have a closer look at smart banners. These are interactive ads with automatic generation and pay per click. This article describes such a format and its potential users, as well as points to be taken into account when creating campaigns. What are smart banners? Smart banners are media ads with dynamic content for search websites and Yandex Advertising Network (YAN). You need a feed to run a campaign. Based on it, Yandex.Direct automatically creates smart banners and smart ads for each product. Smart ads are similar to classic YAN text and image ads, but they are created automatically and suit the interests of a specific user. In addition, smart banners occupy the entire advertising block, and smart ads share it with other advertisers. Smart banners and smart ads Such an advertising campaign is targeted at users who have already viewed products on your website or similar products on competitors' websites. Yandex.Direct analyzes user's requests and shows what it might like. How the audience is selected to show ads? Yandex.Direct offers two principles to select the audience: Retargeting Ads are generated for everyone who has already viewed specific products in your online store. Data on each transaction on the website is sent to Yandex.Metrika, and from there to Yandex.Direct. Therefore, the retargeting principle is more effective if behavior statistics are already collected on the website. Behavioural Ads are shown to users who searched for similar products on the Internet. This principle is suitable in three cases: if statistics have not yet been collected, target the audience of competitors; if you need to attract a new audience; if there are few products and services. Who are smart banners for? The format is available for most online stores and businesses. The subject can be any, if it does not contradict the Yandex.Direct requirements. First of all, smart banners will be useful to large online stores with a large and regularly changing product range. Such ads help to make the offer specific to the user. For example, they show the products forgotten in the cart.However, smart banners are effective for websites with a small range of products or services. If this is your case, choose an audience based on a behavioral principle and show ads to those who are looking for something similar from competitors. In order for any dynamic advertising (including smart banners) to work well, you need a detailed and elaborate product feed. We’ll tell you how to prepare it and what else to do before running the campaign. Getting ready Prepare the feed The feed is a file with all the information about a product range. It contains the main product characteristics: for example, name, description, price, manufacturer, image or link. Here are Yandex’s feed requirements: The data set in the feed determines what information will be displayed in the ad. If the file is partially filled, the necessary information will not be displayed in the ad or the product will not be included in the search results at all. If you have a large online store with a large number of products, enable a file download via the link and configure auto-update. So the range, prices and characteristics will always be relevant. If there are few products and services, create the feed manually or using an online generator. But in this case, you will have to update the file yourself upon all changes. Setting up the advertising campaign Select Smart Banners in the Create Campaign tab in Yandex.Direct. Add a name The name should be clear and simple: it’s easier to navigate when there are several campaigns. Specify a schedule of ad displays and region At the initial stage, set a round-the-clock ad display: this way you will collect more complete statistics. Choose a region where your target audience resides. Choose a strategy Unlike dynamic ads in YAN, smart banners do not have manual rate management, and 3 strategies are offered to advertisers: Conversion Optimization (CPA, Cost Per Click) — maximum targeted clicks per week at a given price. Suitable for starting if statistics are accumulated. Choose a lower target if you have less than 10 purchases per week. Specify how much you are ready to pay for the chosen target. Click Optimization (CPC, Cost Per Click) — maximum clicks per week, taking into account limitations on budget, average cost per click, or target number of clicks. Profitability Optimization (ROI, Return On Investment) — maximum conversion with profitability close to the target. Specify the platforms Smart banners can be displayed on different platforms: search engines, YAN, on both platforms. To track user behavior and collect high-quality statistics, create separate advertising campaigns for different platforms. Stop showing ads in mobile apps You can partially disable ad displays on certain platforms and in external networks. For example, in a mobile app, a user clicks on an ad in order to skip it more quickly, and thereby spends your budget. Create an ad group if you have one feed for all products, create an ad group and configure filters — the product selection criteria to generate smart banners. To cover all products, you can create: many ad groups with a single filter and therefore a single product category; many filters in one group (up to 50 filters). When creating an ad group, specify a clear name, feed and tracking link parameters — dynamic UTM tags. Headers for smart banners or texts for smart ads are automatically generated. But you can change this: specify feed fields to take the necessary headings and descriptions from. Configure filters To configure filters, choose the audience you’d like to display the smart ad group to: Interested in similar products. Ads are shown to those who have viewed similar products on other websites. The ads are displayed both in search engines and in networks. Viewed products on my website. Retargeting by product cards of your online store. The ads are displayed in networks only. Both audiences. In networks, ads are displayed to everyone who has visited your website or searched for similar products on the Internet. In search engines, ads are displayed only to visitors who were interested in similar products. Now configure the criteria to display ads. For example, products of a particular manufacturer or only those in stock. For more information on how to effectively configure filters, see Yandex Help. Create ads At this point, you work on the appearance of your ads. Choose the desired category and the maximum number of formats. In addition to the required fields, specify the ad display interval. Add a display of discounts by checking the “Old price” and “Discount” fields. It happens that the headings or descriptions do not look as you expected. In this case, add the desired text to the feed, and when making creatives, specify a field to take information from. If all the ad formats look good at the preview, your campaign is ready, submit it for moderation. Now track changes in statistics and change creatives, texts and keywords, if necessary. Let’s discuss how you can improve the effectiveness of campaigns. Ad optimization Smart banner campaign analytics are similar to YAN and search engine ads. Build campaign statistics in the Report Wizard. Specify the period, the necessary parameters and select the attribution model. Perhaps your campaign was not the first in the click chain. Stop the campaign in case of high expenses and click rate, but low conversion rate. Track filter performance and adjust your budget. Important: the analysis quality depends on the correct feed and properly configured e-commerce. Using special tools, you can simplify the advertising campaign development. For example, to quickly and efficiently build a semantic core for thousands of products. For such purposes, eLama has a language generator. This is a tool that independently selects a detailed list of key phrases, compiles ad texts and exports created campaigns to the advertising system. Summary In order for smart banners to provide maximum conversion rates, follow the basic rules: test smart banners in various topics, separate search engine advertising campaigns from YAN campaigns, choose a strategy that meets KPI, change automatic headers if they don’t fit, manage ads based on statistics.

How To Set Up Smart Banners in Yandex.Direct

On a , I provided some details on a disappointing test with a few free DNS resolvers that were supposed to block access to phishing and malicious domains. recent post In summary: . To be fair to them, my testing was not very scientific as I only got a few random malicious domains from pishtank, malware domains and my email, and expected them to be blocked. So maybe .. that was expected. they did not work as expected However, that got me thinking .. ? If they were not able to block access to malicious domains, would they be able to block access to porn? I assume the majority of people using these services are actually looking to block access to adult content for their kids. Would they at least be able to do a good job there? What about our children Testing Porn Filters (the free ones) So testing again, I went. I picked up 6 free Porn/Adult/Explicit content filters: * ( 208.67.222.123 ) *Neustar_FamilySecure ( 156.154.70.3 ) * ( 185.228.168.168 ) * ( 199.85.126.20 ) * ( 77.88.8.7)*AdGuard_Family ( 176.103.130.132) OpenDNS_FamilyShield CleanBrowsing_FamilySafe Norton_ConnectSafe Yandex_Family And tried to see if they would be able to block access to porn. I chose 88 different pornographic sites (the first ones ranked on Google & Bing for the keyword "porn"). I pretty much went manually — page by page, getting each domain. After page 6, I skipped to page 15 and kept collecting domains. The reason I skipped to page 15 was to get a few less popular domains. I also re-did the test with two other similar keyword that means "having sexual intercourse" in a less noble ways. That's how I got the . 88 different domains Results Out of the 88 porn domains, I expected all of them to be blocked. They were ranked on the search engines and easily found online. Only blocked them all, with SafeConnect very close in second place by missing 5 domains: CleanBrowsing Norton : 100% blocked : 94% blocked (83 blocked, 5 not blocked) : 93% blocked (82 blocked, 6 not blocked) : 89% blocked (79 blocked, 9 not blocked) : 81% blocked (72 blocked, 16 not blocked) : 55% blocked (55 blocked, 39 not blocked) CleanBrowsing Norton Yandex OpenDNS Neustar AdGuard Neustar and AdGuard were not good at all, missing even some of the popular sites. Full table of results is on this link: https://pastebin.com/raw/ztA7HQVE Partial screenshot of the pastebin with the results (medium doesn't make it easy to import a table): Testing free proxies There are many easy to use free proxies that should be blocked when you are filtering access to pornographic content. When testing the top 10 free proxy domains, only and blocked all of them. OpenDNS CleanBrowsing None of the other services tried to block access to proxies. I would give them a -1 for making it easy to bypass their own filters. Forcing Safe Search One of the things that are important when blocking access to pornographic content is restricting search engines — as they cache porn images on the results, bypassing the filters control. Google offers a way to force Safe Search via DNS and only , and enforced them. That's a big plus for these 3 services for that. Yandex AdGuard CleanBrowsing TLDR If you got here, you read the whole thing, so that's pretty much it. I didn't test any of the paid services, so can't attest on their accuracy. What else can we test? Suggestions are welcome.

Porn Filters Compared: OpenDNS, Neustar, CleanBrowsing, Norton, Yandex and AdGuard

Since my ISP thought was a good idea to modify their resolver responses to push me to their own "search engine", I realized the importance of the free DNS projects that Google, OpenDNS and others are offering. They give you an un-modified DNS experience that you may not get from your ISP. In addition to that, some players also block malicious domains and offer an option to block certain domain categories (mostly adult content) for families and kids. That's all awesome as now we have a variety of options to choose from. However, DNS performance is critical for your own Internet experience, so before you choose a provider, we recommend testing it to see how fast it will load from your location. In this article, we will compare the raw performance of a few popular providers from multiple locations throughout the world and see how they fare against each other. Tested players Unfiltered DNS + Optional Blocking of malicious domains: Google: 8.8.8.8 ( ) unfiltered only Quad9: 9.9.9.9 OpenDNS: 208.67.222.222 Filtered / Safe DNS — Blocking Adult content for families / children: OpenDNS Family Shield: 208.67.222.123 : 185.228.168.168 CleanBrowsing Norton : 199.85.126.20 ConnectSafe : 77.88.8.7 Yandex DNS Locations As far as locations, we tested from these 14 locations from around the globe: : New York, Toronto, Montreal, Atlanta, Dallas, San Francisco North America : London, Paris, Amsterdam, Frankfurt Europe : Tokyo, Singapore, Australia Asia : Sao Paulo South America Results Summary I divided the test into 2 phases to see how it would fare against each other. First, I did the basic ping test. Ping's help us measure latency and distance between our testing servers and their DNS servers. Second, we actually measure the cached DNS lookup for a couple domains: Google.com, Yahoo.com and Medium.com and compared all the players. TLDR: Google was the most stable and with the best average performance across all locations. Quad9 is pretty strong and faster than Google in multiple locations. OpenDNS is still pretty good across the US, but not as fast in Asia and South America. CleanBrowsing and Norton are pretty consistent and with good performance across all locations. Yandex is for Russia only. But see for yourself. Test 1: DNS Lookup The DNS lookup test was done using the utility and extracting the from the results. For each domain we tested it a 5 times, with a few minutes within each one. We removed the worst response and averaged the others (to reduce the chance of a latency from our test location affect the results). dig "Query time" Summary response out of the most locations we tested. It was under 2msec in 8 different locations, which is amazing. Quad9 got the fastest when comparing the averages (9.5msec). Google got in #1 place Quad9 did not perform well across a few cities (specially Sao Paulo and Tokyo) and had an average response time of 26msec, higher than Google (9msec). Yandex was the worst across all locations. was the fastest when offering adult content filtering. CleanBrowsing Average results (when removing the worst location of each) #1 : 9.538 msec#2 : 9.923 msec#3 : 26.07 msec#4 : 32.76 msec#5 : 38.10 msec#6 _Family: 40.69 msec#7 : 160.7 msec Google CleanBrowsing Quad9 Norton OpenDNS OpenDNS Yandex Results #1 Google: 1 msec #1 Quad9: 1 msec #3 CleanBrowsing: 4 msec #4 Norton: 6 msec #5 OpenDNS: 17 msec #6 OpenDNS_Family: 21 msec #7 Yandex: 104 msec NewYork #1 Quad9: 1 msec #2 Google: 10 msec #3 OpenDNS: 25 msec #4 OpenDNS_Family: 28 msec #4 CleanBrowsing: 28 msec #6 Norton: 29 msec #7 Yandex: 123 msec Toronto #1 CleanBrowsing: 16 msec #2 Quad9: 17 msec #3 Google: 18msec #4 Norton: 25 msec #5 OpenDNS: 27 msec #6 OpenDNS_Family: 27 msec #7 Yandex: 116 msec Montreal #1 Quad9: 1 msec #2 Google: 2 msec #3 CleanBrowsing: 19 msec #4 OpenDNS: 21 msec #5 OpenDNS_Family: 30 msec #6 Norton: 24 msec #7 Yandex: 133 msec Atlanta #1 Google: 2 msec #2 CleanBrowsing: 2 msec #3 Norton: 3 msec #4 OpenDNS_Family: 7 msec #5 OpenDNS: 16 msec #6 Quad9: 30 msec #7 Yandex: 147 msec Dallas #1 Norton: 2 msec #2 CleanBrowsing: 9 msec #3 Google: 10 msec #4 Quad9: 26 msec #5 OpenDNS: 40 msec #6 OpenDNS_Family: 40 msec #7 Yandex: 175 msec SanFrancisco #1 Quad9: 1 msec #2 CleanBrowsing: 3 msec #3 Norton: 4 msec #4 Google: 8 msec #5 OpenDNS_Family: 15 msec #6 Yandex: 34 msec #7 OpenDNS: 42 msec London #1 CleanBrowsing: 6 msec #2 Quad9: 7 msec #3 OpenDNS_Family: 10 msec #4 Google: 11 msec #5 Norton: 20 msec #6 OpenDNS: 23 msec #7 Yandex: 39 msec Paris #1 Google: 1 msec #1 Quad9: 1 msec #3 CleanBrowsing: 9 msec #4 Norton: 13 msec #5 OpenDNS: 14 msec #6 OpenDNS_Family: 14 msec #7 Yandex: 29 msec Amsterdam #1 Quad9: 1 msec #2 OpenDNS: 9 msec #3 Google: 10 msec #4 Norton: 11 msec #4 CleanBrowsing: 14 msec #6 OpenDNS_Family: 20 msec #7 Yandex: 35 msec Frankfurt #1 CleanBrowsing: 5 msec #2 Google: 15 msec #3 Norton: 113 msec #4 OpenDNS: 126 msec #5 OpenDNS_Family: 127 msec #6 Quad9: 179 msec #7 Yandex: 238 msec SaoPaulo #1 OpenDNS: 1 msec #2 CleanBrowsing: 5 msec #3 Norton: 8 msec #4 Google: 18 msec #5 OpenDNS_Family: 36 msec #6 Quad9: 65 msec #7 Yandex: 309 msec Tokyo #1 OpenDNS_Family: 1 msec #2 Google: 2 msec #2 Quad9: 2 msec #2 OpenDNS: 2 msec #5 CleanBrowsing: 4 msec #6 Norton: 9 msec #7 Yandex: 269 msec Singapore #1 CleanBrowsing: 5 msec #2 Quad9: 7 msec #3 Google: 16 msec #4 OpenDNS: 131 msec #5 OpenDNS_Family: 153 msec #6 Norton: 159 msec #7 Yandex: 339 msec Sydney Test 2: Ping All those providers, except for Yandex, have an anycast network, so they performed very well. Yandex seems to be restricted to Russia, so had very high ping times compared to the others. We removed them from the results as their ping tests were way above 100ms for all locations. Norton does not allow ICMP, so could not be tested. We consider any response under 5ms to be pretty strong and equal between each other as they are undistinguished for the end user. Responses between 5-20ms are acceptable (OK) and anything between 20ms-100ms we consider to be slow. Ping responses higher than 100ms get a FAIL as it means there is not presence on that location (or somewhere close by). Winners All providers performed really well, but both Google and Quad9 got 1st 5 times out of the 14 locations. CleanBrowsing came out in 3rd, being the faster in 3 locations. However, when we consider the average across allow all locations, Quad9 did not perform as well, as it had very high ping time from Sao Paulo, Tokyo and Dallas. Considering the averages, those were the winners: Google: 2.332 msCleanBrowsing: 3.545 msOpenDNS: 11.93 msOpenDNS_Family: 13.34 msQuad9: 21.73 ms Those number are not very fair with OpenDNS as they performed very well, except for Sao Paulo, which skewed the results. If we remove Sao Paulo the numbers change to: Google: 2.318CleanBrowsing: 3.473OpenDNS: 4.233OpenDNS_Family: 5.675Quad9: 9.941 In any event, go ahead and check for yourself: #1 Quad9: 0.900 ms #2 Google: 1.59 ms #3 CleanBrowsing: 2.98 ms #OK OpenDNS: 9.11 ms #OK OpenDNS_Family: 9.01 ms NewYork #1 Quad9: 0.467 ms #2 OpenDNS: 0.378 ms #3 OpenDNS_Family: 0.379 ms #4 Google: 0.749 ms #5 CleanBrowsing: 1.78 ms Toronto #1 CleanBrowsing: 9.21 ms #2 Google: 9.91 ms #OK OpenDNS: 16.5 ms #OK OpenDNS_Family: 16.5 ms #OK Quad9: 16.6 ms Montreal #1 Quad9: 0.585 ms #2 Google: 0.947 ms #OK CleanBrowsing: 11.2 ms #OK OpenDNS: 12.3 ms #SLOW OpenDNS_Family: 24.2 ms Atlanta #1 OpenDNS: 0.961 ms #2 OpenDNS_Family: 0.905 ms #3 Google: 1.09 ms #4 CleanBrowsing: 1.19 ms #SLOW Quad9: 29.9 ms Dallas #1 CleanBrowsing: 2.13 ms #2 OpenDNS: 2.34 ms #3 OpenDNS_Family: 2.37 ms #4 Quad9: 2.70 ms #5 Google: 2.92 ms SanFrancisco #1 Quad9: 0.936 ms #2 OpenDNS: 1.54 ms #3 CleanBrowsing: 2.18 ms #4 Google: 2.91 ms #5 OpenDNS_Family: 8.12 ms London #1 Google: 4.51 ms #2 CleanBrowsing: 4.53 ms #3 OpenDNS_Family: 4.57 ms #4 OpenDNS: 4.69 ms #5 Quad9: 7.05 ms Paris #1 Google: 0.317 ms #2 OpenDNS: 0.852 ms #3 Quad9: 0.906 ms #4 OpenDNS_Family: 1.62 ms #5 CleanBrowsing: 4.85 ms Amsterdam #1 Google: 1.07 ms #2 CleanBrowsing: 1.29 ms #3 Quad9: 1.37 ms #4 OpenDNS: 1.41 ms #5 OpenDNS_Family: 1.59 ms Frankfurt #1 Google: 2.51 ms #2 CleanBrowsing: 4.49 ms #FAIL OpenDNS: 112 ms #FAIL OpenDNS_Family: 113 ms #FAIL Quad9: 175 ms SaoPaulo #1 CleanBrowsing: 1.15 ms #2 OpenDNS: 1.23 ms #3 OpenDNS_Family: 1.27 ms #4 Google: 1.81 ms #SLOW Quad9: 65.4 ms Tokyo #1 Quad9: 1.25 ms #2 CleanBrowsing: 1.57 ms #3 Google: 1.58 ms #4 OpenDNS_Family: 1.61 ms #5 OpenDNS: 1.92 ms Singapore #1 Google: 0.738 ms #2 CleanBrowsing: 0.992 ms #3 Quad9: 1.18 ms #4 OpenDNS_Family: 1.64 ms #5 OpenDNS: 1.81 ms Sydney

Stories, News & Mentions about YANDEX

Yandex’s Stories on HackerNoon

Yandex's News & Mentions