TYDEi

tydei

Reflection in dotnet is incredibly powerful — but with great power comes great responsibilities. Aside from all sorts of misuse that can arise with reflection, one of the reasons that dotnet reflection gets flack is because of performance. For that reason, I put this article together to talk about a comparison between two popular ways that you can create object instances. This will be the battle of Activator.CreateInstance vs Type.InvokeMember. In this article, I’ll explain how you can use each to create new instances of types and I’ll also provide some benchmark details to explain who comes out on top. Of course, there’s going to be a little surprise. So, let’s dive into reflection in dotnet and compare Activator.CreateInstance vs Type.InvokeMember methods! Understanding Reflection in DotNet DotNet Reflection is a powerful feature that allows you to inspect and manipulate your types at runtime. It provides the ability to dynamically load assemblies, examine and modify their metadata, and create instances of types that are not known at compile-time. I use this all of the time in my . C# applications because I leverage plugin architectures all of the time Reflection plays an important role in many advanced programming scenarios. It enables us to build flexible and extensible software solutions by providing the ability to perform operations on types, methods, properties, and fields that are not known at design-time. It’s also these powerful abilities that allow folks to use it for questionable reasons in their regular development — they hit a wall in their design but reflection allows them to walk right around it. However, I feel that as a C# developer, it’s important to understand Reflection in DotNet because it opens up a whole new world of possibilities. With Reflection, you can build generic frameworks, implement dependency injection, create dynamic proxies, and much more. But these generally aren’t the core of what you’re building — they’re supporting pieces. That’s why they might be a good fit for it compared to the bulk of your business logic using reflection. One of the key benefits of Reflection is its ability to dynamically create instances of types using either or . These methods allow us to create objects without having knowledge of the concrete type at compile-time. In the following sections, we’ll explore these two methods of creating instances using Reflection as we compare vs . Activator.CreateInstance Type.InvokeMember Activator.CreateInstance Type.InvokeMember Creating Instances Using Activator.CreateInstance In C#, the method is one of the most popular ways that developers use to create instances via reflection. It allows the creation of new instances of classes at runtime, even without having knowledge of their specific types beforehand. This method belongs to the class and is commonly used in scenarios where dynamic instantiation is required. Activator.CreateInstance System.Activator The primary purpose of is to dynamically create instances of classes — so if you have access to the type at compile time there’s probably not a good reason for you to be using this! can be particularly useful in situations where the type of object to create is not known until runtime, such as when loading plugins or working with dynamically loaded assemblies. It eliminates the need for hardcoding explicit constructors and provides flexibility in object creation. Activator.CreateInstance Activator.CreateInstance Advantages and Disadvantages of using Activator.CreateInstance Using offers several advantages over normal object instantiation: Activator.CreateInstance Allows for late-bound object creation, which can be beneficial in scenarios where object types can vary at runtime. Can simplify the codebase by eliminating the need for switch statements or if-else conditions to handle different object types. Provides a dynamic and extensible approach to object creation. However, there are also some disadvantages to consider when using : Activator.CreateInstance The performance overhead of using reflection can be higher compared to direct instantiation due to the additional steps involved in resolving types at runtime. relies on the existence of a public parameterless constructor. Otherwise, you need to consistently know which arguments to pass in — challenging if you’re dynamically doing this for many different types. Activator.CreateInstance generally Prone to bugs when the target type is modified because there are no compile-time checks for signature compatibility. Code Examples of Using Activator.CreateInstance The following code examples demonstrate how to use to create instances dynamically: Activator.CreateInstance // Example 1: Creating an instance of a known type
Type objectType = typeof(MyClass);
object instance = Activator.CreateInstance(objectType); In example 1, we use to obtain the object representing the known class . Then, we use to create a new instance of . typeof Type MyClass Activator.CreateInstance MyClass // Example 2: Creating an instance of an unknown type at runtime
string typeName = "MyNamespace.MyClass";
Type unknownType = Type.GetType(typeName);
object dynamicInstance = Activator.CreateInstance(unknownType); In example 2, we have an unknown type represented by a string . We use to obtain the object based on the provided type name. Finally, is used to create a new instance of the dynamically determined type. typeName Type.GetType Type Activator.CreateInstance We’ll look at one more example where we can pass parameters in for the constructor — again, making the assumption that we’ll know the signature since we can’t prove it at compile time via this method: // Example 3: Creating an instance with constructor parameters:
string typeName = "MyNamespace.MyClass";
Type unknownType = Type.GetType(typeName);
Object dynamicInstance = Activator.CreateInstance(
    unknownType,
    new[]
    {
        "Hello World!", //  this is the single string parameter!
    }); Creating Instances Using Type.InvokeMember is a method available to us from Reflection in DotNet that allows us to dynamically create instances of a type. It provides a flexible way to instantiate objects at runtime by utilizing the information about the type at hand. For these reasons, it’s very similar in terms of how you might go leverage it to create instances of objects. Type.InvokeMember Advantages and Disadvantages of using Type.InvokeMember Here are some general advantages to using over normal object instantiation: Type.InvokeMember Allows for late-bound object creation, which can be beneficial in scenarios where object types can vary at runtime. Can simplify the codebase by eliminating the need for switch statements or if-else conditions to handle different object types. Provides a dynamic and extensible approach to object creation. Wait a second… Isn’t this the same list that we saw above for ? That’s right. So let’s cut this part short. We’re not going to see any big difference until we start looking at performance — and perhaps in some very specific edge cases. But overall both provide very comprehensive ways to dynamically instantiate objects and InvokeMember is a bit more verbose since it handles more than just constructors. Activator.CreateInstance Let’s check out some code before hitting the benchmarks. Code Example of Using Type.InvokeMember Here is an example code snippet demonstrating the usage of to create an instance of a type dynamically: Type.InvokeMember // Example 1: Creating an instance of a known type
Type objectType = typeof(MyClass);
var instance = objectType
.InvokeMember(
    null,
    BindingFlags.CreateInstance,
    null,
    null,
    null); In the above example, we first obtain the Type object representing the class “MyClass”. We then use Type.InvokeMember to create an instance of that class and assign it to the “instance” variable. This allows us to create an object of “MyClass” dynamically without explicitly specifying the class name. And to do this without knowing the class at compile-time, it’s very much like before. This part has nothing to do with InvokeMember though: // Example 2: Creating an instance of an unknown type at runtime
string typeName = "MyNamespace.MyClass";
Type unknownType = Type.GetType(typeName);
var instance = objectType.InvokeMember(
    null,
    BindingFlags.CreateInstance,
    null,
    null,
    null); And finally, if we need to pass in some constructor parameters then we can do that as well: // Example 3: Creating an instance with constructor parameters:
string typeName = "MyNamespace.MyClass";
Type unknownType = Type.GetType(typeName);
var instance = objectType.InvokeMember(
    null,
    BindingFlags.CreateInstance,
    null,
    null,
    new[]
    {
        "Hello World!",
    }); Benchmarking Activator.CreateInstance vs Type.InvokeMember Alright — onto the juicy stuff. We’re going to look at to see if aside from API usage we come up with anything different between the two. If you don’t have much experience using BenchmarkDotNet and want to see more, : benchmarking these different approaches you can check out this video on how to use BenchmarkDotNet https://youtu.be/Hqeq9ycqteQ?embedable=true BenchmarkDotNet Setup for Reflection Performance I figured I’d come up with three situations we could run benchmarks against using BenchmarkDotNet: Parameterless constructor class Constructor with a single string parameter Primary constructor with a single string parameter I wanted to toss in primary constructors because I know that’s a feature getting a lot of people up in arms — might as well get some data on it! Here are the classes we’ll be instantiating, for reference: public class ParameterlessClass
{
}

public class ClassicStringParameterClass
{
    private readonly string _value;

    public ClassicStringParameterClass(string value)
    {
        _value = value;
    }
}

public class PrimaryConstructorStringParameterClass(
    string _value)
{
} As for benchmarks, let’s check out the following classes that we’ll be running. Keep in mind that I am using as the baseline because I want to compare vs — I am only including the normal constructor pathway just as a reference. You can : Activator.CreateInstance Activator.CreateInstance Type.InvokeMember find all of this code on GitHub as well [ShortRunJob]
public class ParameterlessClassBenchmarks
{
    private Type? _type;

    [GlobalSetup]
    public void GlobalSetup()
    {
        _type = typeof(ParameterlessClass);
    }

    [Benchmark]
    public void Constructor()
    {
        var instance = new ParameterlessClass();
    }

    [Benchmark(Baseline = true)]
    public void Activator_Create_Instance()
    {
        var instance = Activator.CreateInstance(_type!);
    }

    [Benchmark]
    public void Type_Invoke_Member()
    {
        var instance = _type!.InvokeMember(
            null,
            BindingFlags.CreateInstance,
            null,
            null,
            null);
    }
}

[ShortRunJob]
public class ClassicStringParameterClassBenchmarks
{
    private Type? _type;

    [GlobalSetup]
    public void GlobalSetup()
    {
        _type = typeof(ClassicStringParameterClass);
    }

    [Benchmark]
    public void Constructor()
    {
        var instance = new ClassicStringParameterClass("Hello World!");
    }

    [Benchmark(Baseline = true)]
    public void Activator_Create_Instance()
    {
        var instance = Activator.CreateInstance(
            _type!,
            new[]
            {
                "Hello World!",
            });
    }

    [Benchmark]
    public void Type_Invoke_Member()
    {
        var instance = _type!
            .InvokeMember(
                null,
                BindingFlags.CreateInstance,
                null,
                null,
                new[]
                {
                    "Hello World!",
                });
    }
}

[ShortRunJob]
public class PrimaryConstructorStringParameterClassBenchmarks
{
    private Type? _type;

    [GlobalSetup]
    public void GlobalSetup()
    {
        _type = typeof(PrimaryConstructorStringParameterClass);

    }

    [Benchmark]
    public void Constructor()
    {
        var instance = new PrimaryConstructorStringParameterClass("Hello World!");
    }

    [Benchmark(Baseline = true)]
    public void Activator_Create_Instance()
    {
        var instance = Activator.CreateInstance(
            _type!,
            new[]
            {
                "Hello World!",
            });
    }

    [Benchmark]
    public void Type_Invoke_Member()
    {
        var instance = _type!
            .InvokeMember(
                null,
                BindingFlags.CreateInstance,
                null,
                null,
                new[]
                {
                    "Hello World!",
                });
    }
} Activator.CreateInstance vs Type.InvokeMember: Who is the Champion?! When we pit these two Reflection methods head to head, the winner is… situational. In one of the most common cases, I’d say there’s a very clear winner but for the others, they’re very close. But make sure you read the conclusion because this isn’t the end of the story. The first benchmark we’re going to look at is for a parameterless constructor: Clear winner here: , almost by an order of magnitude. If you don’t have any parameters on your constructor, your best bet is this one. Activator.CreateInstance Next, let’s check out vs for a constructor taking in a single string parameter: Activator.CreateInstance Type.InvokeMember The winner? Not so obvious. These are basically neck-and-neck here, and even though comes out a smidge ahead, it’s nearly negligible. Activator.CreateInstance The last scenario to look at is going to be for primary constructors, and in this case, a primary constructor that takes in a single string parameter: The winner: , but only by a little bit. Very interesting that this is the reverse of what we saw in the previous benchmark! Type.InvokeMember Wrapping Up Activator.CreateInstance vs Type.InvokeMember When it comes to the performance results of vs , there’s a clear winner for the parameterless constructor case: . But when we get into requiring parameters or using primary constructors with parameters, it starts to even out or even favor . Activator.CreateInstance Type.InvokeMember Activator.CreateInstance Type.InvokeMember But this is only the FIRST part of the picture… There’s one more scenario we’ll look at that is the *true* winner, which you can . It continues with these examples and provides updated benchmarks for you as well. read all about in this article here If you found this useful and you’re looking for more learning opportunities, consider and check out my ! subscribing to my free weekly software engineering newsletter free videos on YouTube Also published here.

Head to Head Performance of Activator.CreateInstance vs Type.InvokeMember

Kafka is widely used for building event-driven architectures, allowing services to communicate through messages. However, when working with a foreign provider that adds a " " header to messages, you may encounter serialization and deserialization issues. microservices TypeId This article will guide you through the process of configuring a custom type mapper in Spring Kafka to handle the mapping of types between the producer and consumer services. By implementing this approach, you can successfully consume messages with different package structures and types, avoiding exceptions caused by incompatible types and ensuring smooth message processing. Understanding the deserialization Exception with " " Header TypeId In scenarios where the classes used in the consumer service have a different package structure compared to the producer service's DTO classes, the serialization exception can occur due to the mismatched class paths. DTO The serialization process relies on both the producer and the consumer having access to the same class definitions. When the consumer receives a message with a " " header referencing a class that resides in a different package than the consumer's DTO classes, the deserialization process fails to locate the appropriate class definition. This results in the "ClassNotFoundException" or "IllegalArgumentException" within the serialization exception. TypeId These exceptions can disrupt the smooth consumption of messages. Understanding the root cause of this exception is key to effectively handling the situation. An example of the serialization exception is as follows: javaCopy codejava.lang.IllegalStateException: This error handler cannot process 'SerializationException's directly; please consider configuring an 'ErrorHandlingDeserializer' in the value and/or key deserializer
	at org.springframework.kafka.listener.DefaultErrorHandler.handleOtherException(DefaultErrorHandler.java:198) ~[spring-kafka-3.0.7.jar:3.0.7]
Caused by: org.apache.kafka.common.errors.RecordDeserializationException: Error deserializing key/value for partition consumer-test-18 at offset 1. If needed, please seek past the record to continue consumption.
	at org.apache.kafka.clients.consumer.internals.Fetcher.parseRecord(Fetcher.java:1438) ~[kafka-clients-3.4.0.jar:na]
Caused by: java.lang.IllegalArgumentException: The class 'org.example.dto.out.OutMessageImplTwo' is not in the trusted packages: [java.util, java.lang, org.example.dto.in, org.example.dto.in.*]. 
If you believe this class is safe to deserialize, please provide its name. If the serialization is only done by a trusted source, you can also enable trust all (*).
	at org.springframework.kafka.support.mapping.DefaultJackson2JavaTypeMapper.getClassIdType(DefaultJackson2JavaTypeMapper.java:129) ~[spring-kafka-3.0.7.jar:3.0.7] The exception message provides insights into the underlying problem. In this case, the exception occurs because the class is not present in the list of trusted packages. The trusted packages serve as a safeguard to ensure secure deserialization and prevent unauthorized classes from being deserialized. org.example.dto.out.OutMessageImplTwo When deserializing a message, the default behavior of Spring Kafka is to check if the class specified in the "TypeId" header is within the trusted packages. If it is not found, an exception is thrown. This mechanism is in place to protect against potential security risks, as deserializing untrusted classes can lead to code execution vulnerabilities. To resolve this issue, you need to ensure that the consumer can properly map the incoming message to the corresponding class, even if they have different package structures. One approach is to configure a custom type mapper that explicitly handles the type mapping between the " " header and the consumer's DTO classes. TypeId By implementing a custom type mapper, you can instruct the consumer to map the " " header to the appropriate class or even ignore this header. This enables successful deserialization and ensures that the consumer can process the message correctly. TypeId In the upcoming sections, we will delve into the details of configuring and implementing a custom type mapper in Kafka listeners, allowing seamless message processing and addressing the challenges encountered when the consumer's DTO classes differ from those of the producer. Implementation of custom-type mapper In our example, we will send an implementation of OutMessageAbstract to Kafka and will read it as an implementation of the InMessageAbstract class. These classes are absolutely identical, they differ in name and package. The figure below shows the project structure. Our KafkaListener looks like this. It implements the logic for processing each type of message from the topic. @Component
@Log
@KafkaListener(
        topics = "${spring.cloud.stream.bindings.topic-in-0.destination}",
        groupId = "${spring.cloud.stream.bindings.topic-in-0.group}",
        containerFactory = "kafkaListenerContainerFactory")
public class EventListeners {
    @KafkaHandler
    public void handleInMessageImplEvent(@Payload InMessageImpl event) {
        System.out.println(event);
    }

    @KafkaHandler
    public void handleInMessageImplTwoEvent(@Payload InMessageImplTwo event) {
        System.out.println(event);
    }
} We use the kafkaListenerContainerFactory bean to configure our listener. This factory is described in KafkaConsumerConfiguration and looks like this: @Configuration
public class KafkaConsumerConfiguration {
    private final String SERVER;
    private final String SERVER_PORT;

    public KafkaConsumerConfiguration(
            @Value("${spring.cloud.stream.kafka.binder.brokers}") String server,
            @Value("${spring.cloud.stream.kafka.binder.defaultBrokerPort}") String port
    ) {
        this.SERVER = server;
        this.SERVER_PORT = port;
    }

    private <T> ConsumerFactory<String, T> typeConsumerFactory(Class<T> clazz) {
        Map<String, Object> props = new HashMap<>();
        props.put(ConsumerConfig.BOOTSTRAP_SERVERS_CONFIG, String.format("%s:%s", SERVER, SERVER_PORT));

        return new DefaultKafkaConsumerFactory<>(
                props,
                new StringDeserializer(),
                new JsonDeserializer<>(clazz));
    }

    private <T> ConcurrentKafkaListenerContainerFactory<String, T> initFactory(Class<T> clazz) {
        ConcurrentKafkaListenerContainerFactory<String, T> factory =
                new ConcurrentKafkaListenerContainerFactory<>();
        factory.setConsumerFactory(typeConsumerFactory(clazz));
        return factory;
    }

    @Bean
    public ConcurrentKafkaListenerContainerFactory<String, ?> kafkaListenerContainerFactory() {
        return initFactory(InMessageAbstract.class);
    }
} Because we're adding a “Typeld” header to the message, we'll get the error described above when we try to deserialize. However, we can configure the JsonDeserializer manually. To do this, we must define our implementation of TypeMapper. The following shows how you can define your own TypeMappers. class CustomTypeMapper<T> extends DefaultJackson2JavaTypeMapper {
    private final Class<T> clazz;

    public CustomTypeMapper(Class<T> clazz) {
        this.clazz = clazz;
    }

    @Override
    public JavaType toJavaType(Headers headers) {
        try {
            return TypeFactory.defaultInstance()
                    .constructType(ClassUtils.forName(clazz.getName(), getClassLoader()));
        } catch (ClassNotFoundException e) {
            throw new RuntimeException(e);
        }
    }
} If we add our TypeMapper implementation to the JsonDeserializer when we create the factory, then we can force the Json to be deserialized to the class we need, ignoring the header. The corrected method would look like this: private <T> ConsumerFactory<String, T> typeConsumerFactory(Class<T> clazz) {
    Map<String, Object> props = new HashMap<>();
    props.put(ConsumerConfig.BOOTSTRAP_SERVERS_CONFIG, String.format("%s:%s", SERVER, SERVER_PORT));

    JsonDeserializer<T> jsonDeserializer = new JsonDeserializer<>(clazz);
    jsonDeserializer.setTypeMapper(new CustomTypeMapper<>(clazz));

    return new DefaultKafkaConsumerFactory<>(
            props,
            new StringDeserializer(),
            jsonDeserializer);
} Conclusion By utilizing a custom type mapper in Spring Kafka, you can overcome the challenges associated with consuming messages from foreign providers that include a " " header. This article has provided a step-by-step guide to configuring and implementing the custom type mapper, enabling smooth message processing and avoiding serialization and deserialization exceptions. With this knowledge, you can confidently integrate with external systems and consume Kafka messages with different package structures. TypeId The full project code is available on GitHub: https://github.com/stepanovD/kafka-templates/tree/master/consumer-custom-type-mapper

Handling Custom Type Mapping in Kafka Listeners for Messages with "TypeId" Header

Improving Text Embeddings with
Large Language Models: Conclusion and References

How the IBIS Framework Manages Dataset, License, and AI Model Attributes

This paper is under CC 4.0 license. available on arxiv Authors: (1) Jinrui Yang, School of Computing & Information Systems, The University of Melbourne (Email: jinruiy@student.unimelb.edu.au); (2) Timothy Baldwin, School of Computing & Information Systems, The University of Melbourne and Mohamed bin Zayed University of Artificial Intelligence, UAE (Email: (tbaldwin,trevor.cohn)@unimelb.edu.au); (3) Trevor Cohn, School of Computing & Information Systems, The University of Melbourne. Table of Links Abstract and Intro Background and Related Work Multi-EuP Experiments and Findings Language Bias Discussion Conclusion, Limitations, Ethics Statement, Acknowledgements, References, and Appendix 6 Conclusion In this paper, we introduce Multi-EuP, a novel dataset for multilingual information retrieval across 24 languages, collected from European Parliament debates. The demographic information provided by the Multi-EuP dataset serves a dual purpose: not only does it contribute to multilingual retrieval tasks, but it also holds significant potential for advancing research in the realm of fairness and bias. This dataset can play a pivotal role in investigating issues of equitable representations and mitigation of biases within document ranking settings. Multi-EuP facilitates diverse information retrieval (IR) scenarios, encompassing one-vs-one, one-vs-many, and many-vs-many settings. We demonstrated the utility of Multi-EuP as a benchmark for evaluating both monolingual and multilingual IR. Our study reveals the presence of language bias in multilingual IR when employing BM25. We further validate the effectiveness of mitigating this bias through the strategic implementation of whitespace as a language tokenizer. We propose to conduct future work in three main areas. First, we intend to expand our investigation of language bias to encompass a broader range of ranking methods, including neural methods such as mDPR (Zhang et al., 2021), mColBERT (Lawrie et al., 2023) and PLAID-X(Santhanam et al., 2022). Second, we will expand the dataset by developing an automated API to retrieve data published by the European Parliament (EP), thereby ensuring realtime synchronization of our dataset. Lastly, our current experiments have explored language bias only, but we plan to further investigate gender bias, age bias, and nationality bias. Limitations The limitations of the Multi-EuP dataset are notable but navigable. Primarily, the temporal coverage of the dataset is confined to the past three years. This temporal constraint arises due to the fact that, preceding 2020, documents released by the EU were predominantly available in mono-lingual versions only. However, a potential remedy lies in the amalgamation of the Europarl (Koehn, 2005) collection, enabling a more comprehensive and holistic MultiEuP dataset. Furthermore, it is worth noting the domain skew of the dataset, in that Multi-EuP inevitably centers on political matters. While this presents challenges, particularly in terms of the intricate nuances of political language, it inherently serves as an excellent foundational stepping stone for delving into the intricacies of multilingual retrieval. We believe, however, that this dataset can serve as a launching pad for broader explorations encompassing crossdomain and open-domain transfer learning scenarios, thus contributing to the broader landscape of language understanding and retrieval. Ethics Statement The dataset contains publicly-available EP data that does not include personal or sensitive information, with the exception of information relating to public officeholders, e.g., the names of the active members of the European Parliament, European Council, or other official administration bodies. The collected data is licensed under the Creative Commons Attribution 4.0 International licence. [8] Acknowledgements This research was funded by Melbourne Research Scholarship and undertaken using the LIEF HPCGPGPU Facility hosted at the University of Melbourne. This facility was established with the assistance of LIEF Grant LE170100200. We would like to thank George Buchanan for providing valuable feedback. References Luiz Henrique Bonifacio, Israel Campiotti, Roberto de Alencar Lotufo, and Rodrigo Frassetto Nogueira. 2021. mMARCO: A multilingual version of MS MARCO passage ranking dataset. CoRR, abs/2108.13897. Ilias Chalkidis, Manos Fergadiotis, and Ion Androutsopoulos. 2021. MultiEURLEX - a multi-lingual and multi-label legal document classification dataset for zero-shot cross-lingual transfer. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 6974–6996, Online and Punta Cana, Dominican Republic. Association for Computational Linguistics. 8 https://eur-lex.europa.eu/cont Jonathan H. Clark, Eunsol Choi, Michael Collins, Dan Garrette, Tom Kwiatkowski, Vitaly Nikolaev, and Jennimaria Palomaki. 2020. TyDi QA: A benchmark for information-seeking question answering in typologically diverse languages. Transactions of the Association for Computational Linguistics, 8:454–470. Vladimir Karpukhin, Barlas Oguz, Sewon Min, Patrick Lewis, Ledell Wu, Sergey Edunov, Danqi Chen, and Wen-tau Yih. 2020. Dense passage retrieval for opendomain question answering. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 6769–6781, Online. Association for Computational Linguistics. Omar Khattab and Matei Zaharia. 2020. Colbert: Efficient and effective passage search via contextualized late interaction over BERT. CoRR, abs/2004.12832. Philipp Koehn. 2005. Europarl: A parallel corpus for statistical machine translation. In Proceedings of Machine Translation Summit X: Papers, pages 79–86, Phuket, Thailand. Dawn Lawrie, Eugene Yang, Douglas W. Oard, and James Mayfield. 2023. Neural approaches to multilingual information retrieval. arXiv cs.IR 2209.01335. Jimmy Lin, Xueguang Ma, Sheng-Chieh Lin, JhengHong Yang, Ronak Pradeep, and Rodrigo Nogueira. 2021. Pyserini: A Python toolkit for reproducible information retrieval research with sparse and dense representations. https://github.com/ castorini/pyserini. Tri Nguyen, Mir Rosenberg, Xia Song, Jianfeng Gao, Saurabh Tiwary, Rangan Majumder, and Li Deng. 2016. MS MARCO: A human generated machine reading comprehension dataset. CoRR, abs/1611.09268. Ella Rabinovich, Raj Nath Patel, Shachar Mirkin, Lucia Specia, and Shuly Wintner. 2017. Personalized machine translation: Preserving original author traits. In Proceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics: Volume 1, Long Papers, pages 1074–1084, Valencia, Spain. Association for Computational Linguistics. Razieh Rahimi, Azadeh Shakery, and Irwin King. 2015. Multilingual information retrieval in the language modeling framework. Information Retrieval Journal, 18:246–281. Keshav Santhanam, Omar Khattab, Christopher Potts, and Matei Zaharia. 2022. PLAID: An efficient engine for late interaction retrieval. In Proceedings of the 31st ACM International Conference on Information & Knowledge Management, CIKM ’22, page 1747–1756, New York, NY, USA. Association for Computing Machinery. Jörg Tiedemann and Santhosh Thottingal. 2020. OPUSMT – building open translation services for the world. In Proceedings of the 22nd Annual Conference of the European Association for Machine Translation, pages 479–480, Lisboa, Portugal. European Association for Machine Translation. Eva Vanmassenhove and Christian Hardmeier. 2018. Europarl datasets with demographic speaker information. In Proceedings of the 21st Annual Conference of the European Association for Machine Translation, page 391, Alicante, Spain. Denny Vrandeciˇ c and Markus Krötzsch. 2014. ´ Wikidata: A free collaborative knowledge base. Communications of the ACM, 57:78–85. Konrad Wojtasik, Vadim Shishkin, Kacper Wołowiec, Arkadiusz Janz, and Maciej Piasecki. 2023. BEIRPL: Zero shot information retrieval benchmark for the Polish language. arXiv cs.IR 2305.19840. Peilin Yang, Hui Fang, and Jimmy Lin. 2017. Anserini: Enabling the use of lucene for information retrieval research. In Proceedings of the 40th International ACM SIGIR Conference on Research and Development in Information Retrieval, pages 1253–1256. Xinyu Zhang, Xueguang Ma, Peng Shi, and Jimmy Lin. 2021. Mr. TyDi: A multi-lingual benchmark for dense retrieval. arXiv cs.CL 2108.08787. A. Appendix [8] https://eur-lex.europa.eu/content/ legal-notice/legal-notice.html

Multi-EuP: Analysis of Bias in Information Retrieval - Conclusion, Limitations, and Ethics Statement

This paper is under CC 4.0 license. available on arxiv Authors: (1) Jinrui Yang, School of Computing & Information Systems, The University of Melbourne (Email: jinruiy@student.unimelb.edu.au); (2) Timothy Baldwin, School of Computing & Information Systems, The University of Melbourne and Mohamed bin Zayed University of Artificial Intelligence, UAE (Email: (tbaldwin,trevor.cohn)@unimelb.edu.au); (3) Trevor Cohn, School of Computing & Information Systems, The University of Melbourne. Table of Links Abstract and Intro Background and Related Work Multi-EuP Experiments and Findings Language Bias Discussion Conclusion, Limitations, Ethics Statement, Acknowledgements, References, and Appendix 4 Experiments and Findings We conduct preliminary experiments in both one-vs-one and one-vs-many settings, as described above. We base our experiments on BM25 with default settings (k1 = 0.9 and b = 0.4), a popular traditional information retrieval baseline. Our implementation is based on Pyserini (Lin et al., 2021), which is built upon Lucene (Yang et al., 2017). Notably, the latest LUCENE 8.5.1 API offers language-specific tokenizers, [6] covering 19 out of the 24 languages present in Multi-EuP. For the remaining languages — namely Polish (PL), Croatian (HR), Slovak (SK), Slovenian (SL), and Maltese (MT) — we use a whitespace tokenizer. Methods MRR@k denotes MRR computed at a depth of results. Note that the higher the number the better, and that a perfect retriever achieves an MRR of 1 (assuming every query has at least one relevant document). The choice of setting = 100 aligns with prior endeavors over MS MARCO (Nguyen et al., 2016). k k 4.1 Monolingual IR ( ) one-vs-one We first present results over Multi-EuP in a monolingual setting across the 24 different languages. Specifically, we evaluate single-language queries against documents in the same language. In this configuration, we partitioned our original collection of 22K documents into 24 distinct language-specific sub-collections. Table 2 presents the results broken down across languages. Experimental Setup Table 2 presents the MRR@100 results for BM25 on Multi-EuP. There are two high-level findings: Results and Findings First, Multi-EuP is a relatively easy benchmark for monolingual information retrieval, as the MRR@100 is always around 40 or greater (meaning that the first relevant document is in the top-3 results on average). Indeed, the average MRR across the 24 test languages is 49.61. While direct comparison is not possible, it is noteworthy that for Mr. TYDI, the average MRR is 32.1 across 11 languages. Part of this difference can be attributed to the fact that our relevance judgments are not as sparse as theirs. Second, similar to Mr. TYDI, direct comparison of absolute scores between languages is not meaningful in a monolingual setting, as the document collection size differs. 4.2 Multilingual IR ( ) one-vs-many In contrast to Mr. TYDI (Zhang et al., 2021), Multi-EuP supports one-vs-many retrieval, and allows us to systematically explore the effect of querying the same document collection with the same set of topics in different languages. This is because we have translations of the topics in all languages, documents span multiple languages, and judgments are cross-lingual (e.g., English queries potentially yield relevant Polish documents). For this experiment, we use the default whitespace tokenizer in the Pyserini library. Experimental Setup Results and Findings Table 2 presents the MRR results for BM25 for multilingual information retrieval on 100 topics from the Multi-EuP test set. It’s worth noting that these topics have translationequivalent content in the different languages. Consequently, the one-vs-many approach allows us to analyze language bias. We made several key observations: First, unsurprisingly, having more relevance judgments tends to improve ranking accuracy. Therefore, when comparing English topics with other languages, English exhibits notably better MRR performance. Second, despite there being consistency in the topics, document collection, and relevance judgments, there is a significant disparity in MRR scores across languages, an effect we investigate further in the next section. [6] Provided by the Anaylzer package in LUCENE. https: //lucene.apache.org/core/8_5_1/analyzers-common/ index.html

Multi-EuP: Analysis of Bias in Information Retrieval - Experiments and Findings

This paper is under CC 4.0 license. available on arxiv Authors: (1) Jinrui Yang, School of Computing & Information Systems, The University of Melbourne (Email: jinruiy@student.unimelb.edu.au); (2) Timothy Baldwin, School of Computing & Information Systems, The University of Melbourne and Mohamed bin Zayed University of Artificial Intelligence, UAE (Email: (tbaldwin,trevor.cohn)@unimelb.edu.au); (3) Trevor Cohn, School of Computing & Information Systems, The University of Melbourne. Table of Links Abstract and Intro Background and Related Work Multi-EuP Experiments and Findings Language Bias Discussion Conclusion, Limitations, Ethics Statement, Acknowledgements, References, and Appendix 3 Multi-EuP In our approach, we consider the debate topics to be the queries, and the text of each individual speech delivered by an MEP to be a document. The topics are officially annotated by the EU, and professionally translated into 24 different languages.[5] During preprocessing, we filter out procedural debate topics such as , leaving 1.1K unique topics. They will serve as a valuable resource for assessing language bias in multilingual ranking methods, given that all the topics across different languages are semantically consistent. Topics agenda The 22K multilingual documents within the Multi-EuP dataset originate from MEP speeches during parliamentary debates. Each document annotated with additional metadata, including the date of the speech, the MEP ID, and a link to the video recording for potential multimodal research but not used here. Table 1 shows a detailed breakdown of the language distribution and descriptive statistics of the dataset. We include in our corpus documents only in the original language, as spoken by the MEP, but not their translations into other languages. Our only use of translations is the debate topics themselves. Documents To assess the relevance of documents to a given query, we use a binary relevance judgment, based on whether the speech was part of a debate on the given topic, resulting in one positive relevance judgment per document, meaning that the document collection is much less sparse than Mr. TYDI and MS MARCO, for example. Judgments Multi-EuP covers 24 EU languages from seven families (Germanic, Romance, Slavic, Uralic, Baltic, Semitic, Hellenic), each of which is the official language of one or more member states. Table 1 provides a breakdown of each language’s EU usage, member state distribution, and population, using ISO-639 codes. Languages Multi-EuP encompasses 705 members elected across the 27 member states of the EU. We constructed the MEP dictionary by collecting MEP attributes such as and . We further annotated MEPs with gender and their birthdate, based on Wikipedia profiles and Rabinovich et al. (2017), and manually checked if difference existing. Figure 1 illustrates the gender and age distribution across MEPs, with male MEPs being more than twice as numerous as female MEPs, and the majority falling within the 40– 70 age range. This corpus is rare, perhaps unique, due to its richly detailed speaker demographic information, which enables research on fairness and bias in information retrieval. MEP name, photo, id in EU, nationality, place of birth, party affiliation, spoken language For data splitting, we select two sets with 100 language-specific and distinct topics for development and test set in 24 languages, and keep the remaining topics to the training set. This design choice was made to maintain an ample supply of topics and judgment samples essential for the training of deep learning models, and also facilitate subsequent cross-lingual comparative research. Data Split Similarly to Mr.TYDI (Zhang et al., 2021), Multi-EuP can be used for monolingual retrieval in English as well as non-English languages (eg. Swedish queries against Swedish documents). However, unlike Mr.TYDI, Multi-EuP encompasses multilingual documents and identical multilingual topics, ensuring that queries in different languages can be compared. Consequently, Multi-EuP can support diverse information retrieval experimental tasks. These including scenarios with single one language queries against single one language documents, in other words, monolingual or cross-lingual IR, scenarios with single-language queries against multilingual documents, i.e., multilingual IR, and scenarios involving multilingual queries against multilingual documents, i.e, mixed multilingual IR). Supported Task one-vs-one one-vs-many many-vs-many [5] https://www.europarl.europa.eu/translation/ en/translation-at-the-european-parliament/

Multi-EuP: Analysis of Bias in Information Retrieval - Multi-EuP Use

This paper is under CC 4.0 license. available on arxiv Authors: (1) Jinrui Yang, School of Computing & Information Systems, The University of Melbourne (Email: jinruiy@student.unimelb.edu.au); (2) Timothy Baldwin, School of Computing & Information Systems, The University of Melbourne and Mohamed bin Zayed University of Artificial Intelligence, UAE (Email: (tbaldwin,trevor.cohn)@unimelb.edu.au); (3) Trevor Cohn, School of Computing & Information Systems, The University of Melbourne. Table of Links Abstract and Intro Background and Related Work Multi-EuP Experiments and Findings Language Bias Discussion Conclusion, Limitations, Ethics Statement, Acknowledgements, References, and Appendix 2 Background and Related Work The European Parliament (EP) serves as an important forum for political debates and decisionmaking at the European Union level. Members of the European Parliament (MEP) are elected in direct elections across the EU. The European Parliament debate is presided over by the President, who guides MEPs in discussing specific subjects. EP debates have been the source of three key datasets. First, was crafted by Koehn (2005) by collecting EP debates documents from 1996 to 2011, and extracting translations as a parallel corpus for statistical machine translation, enriched with attributes including , and Europarl-2005 debate date chapter id, MEP id, language, MEP name, MEP party. Later, Rabinovich et al. (2017) built upon , by introducing additional demographic attributes: MEP gender and . These were sourced from sources such as Wikidata (Vrandeciˇ c and Krötzsch ´ , 2014) and automatic annotation tools such as *Genderize[*2] and . [3] However, is limited to only two language pairs: English– German and English–French. (Vanmassenhove and Hardmeier, 2018) expanded upon to add twenty additional language pairs, based on the manual translations in the EP archives. These corpora have been used primarily for machine translation research. Europarl2017 Europarl-2005 MEP age AlchemyVision Europarl-2017 Europarl-2018 Europarl-2017 Since 2020, the EU has publicly released raw debates in the form of transcribed source-language speeches with rich multilingual topic index data, along with the original video and audio recordings. This forms the basis of the Multi-EuP dataset, with additional attributes for each speaking MEP such as an image, birthplace, and nationality. Zhang et al. (2021) introduced Mr. TYDI, an evaluation benchmark dataset for dense retrieval assessment over 11 languages. This dataset is constructed from TYDI (Clark et al., 2020), a question answering dataset. For each language, annotators assign relevance scores as judgments for questions, derived from Wikipedia articles. Notably, the questions for different languages are crafted independently, and relevance judgements are provided in-language only. Based on the dataset, the authors evaluate on monolingual retrieval tasks for non-English languages using BM25 and mDPR as zero-shot baselines. However, Mr. TYDI’s scope is limited in that it is not truly multilingual, in that queries in a given language are only performed over documents in that language. This is part of the void our work aims to address. MS MARCO (Nguyen et al., 2016) is a widelyused dataset, sourced from Bing’s search query logs, but for English queries and documents only. To mitigate this, Bonifacio et al. (2021) introduced mMARCO, a multilingual variant of the MS MARCO passage ranking dataset, spanning 13 languages and created through machine translation, based on one open-source approach (Tiedemann and Thottingal, 2020) and one commercial system in the form of Google Translate.[4] Analysis of the authors’ results reveals a positive correlation between translation quality and retrieval performance, with higher translation BLEU scores yielding improved retrieval MRR outcomes. However, similar to Mr. TYDI, mMARCO focuses on in-language retrieval only for multiple languages, rather than multilingual retrieval. Throughout the past few decades, numerous datasets and tasks pertaining to multilingual retrieval have been developed for evaluation, through efforts such as CLEF, TREC, and FIRE, each contributing standardized document collections and evaluation procedures. These evaluation datasets facilitate genuine multilingual IR research such as Rahimi et al. (2015) and Lawrie et al. (2023). However, the scope of these datasets is generally limited to a small number of queries. For example, in the case of CLEF 2001-2003, each edition encompasses a mere few dozen queries. This limitation tends to confine research predominantly to evaluation and not offer a resource for training a multilingual ranking model. Our dataset is of a scale to accommodate both large-scale training and evaluation of multilingual retrieval methods. Compared with the related work above, our work augments the multilingual mixture of queries and documents compared to Mr.TYDI, preserves the authenticity of multilingual contexts compared to mMARCO’s translation-based approach, and surpasses the query count limitations of tasks like CLEF. [2] https://genderize.io/ [3] https://www.ibm.com/smarterplanet/us/en/ ibmwatson/developercloud/alchemy-vision.html [4] https://cloud.google.com/translate

Multi-EuP: Analysis of Bias in Information Retrieval - Background and Related Work

This paper is under CC 4.0 license. available on arxiv Authors: (1) Jinrui Yang, School of Computing & Information Systems, The University of Melbourne (Email: jinruiy@student.unimelb.edu.au); (2) Timothy Baldwin, School of Computing & Information Systems, The University of Melbourne and Mohamed bin Zayed University of Artificial Intelligence, UAE (Email: (tbaldwin,trevor.cohn)@unimelb.edu.au); (3) Trevor Cohn, School of Computing & Information Systems, The University of Melbourne. Table of Links Abstract and Intro Background and Related Work Multi-EuP Experiments and Findings Language Bias Discussion Conclusion, Limitations, Ethics Statement, Acknowledgements, References, and Appendix Abstract We present Multi-EuP, a new multilingual benchmark dataset, comprising 22K multilingual documents collected from the European Parliament, spanning 24 languages. This dataset is designed to investigate fairness in a multilingual information retrieval (IR) context to analyze both language and demographic bias in a ranking context. It boasts an authentic multilingual corpus, featuring topics translated into all 24 languages, as well as cross-lingual relevance judgments. Furthermore, it offers rich demographic information associated with its documents, facilitating the study of demographic bias. We report the effectiveness of Multi-EuP for benchmarking both monolingual and multilingual IR. We also conduct a preliminary experiment on language bias caused by the choice of tokenization strategy. 1 Introduction Information retrieval (IR) classically uses a retrieval model to query a document collection and return a ranked list of documents which are predicted to be (decreasingly) relevant to the query. Retrieval models have increasingly been based on supervised learning, involving the annotation of documents with relevance scores relative to a given query, and the training of models to predict the relative association between a query and document (Karpukhin et al., 2020; Khattab and Zaharia, 2020). In parallel with these advances, the democratisation of the internet has led to a surge of individual contributors serving as information disseminators, hailing from various countries and regions, and posting in different languages. This has created possibilities for exploration of cross-lingual and multilingual text retrieval. Cross-lingual retrieval pertains to scenarios where queries are formulated in one language but documents are retrieved from another language. On the other hand, multilingual retrieval involves a query in one language but retrieval of documents across multiple languages simultaneously. An important consideration in any such work is both robustness and fairness across different combinations of languages – for instance, are results from one language consistently ranked higher than another for certain types of query. While progress towards multilingual retrieval through the release of datasets such as Mr. TYDI (Zhang et al., 2021) and mMARCO (Bonifacio et al., 2021), both are limited in that they evaluate monolingual retrieval for a range of languages, rather than true multilingual retrieval, using multiple languages simultaneously. Additionally, mMARCO was created by machine translation of MS MARCO (Nguyen et al., 2016), introducing a confounding factor of translation errors. We present a multilingual dataset based on the European Parliament debate archive with queries in 24 distinct languages, and relevance judgements also across all 24 languages. This ensures the “multilingual” nature of the dataset in terms of both query-to-document and document-to-query associations. We additionally augment each document with comprehensive metadata of the author, including gender, nationality, political affiliation, and age, for use in exploring fairness with respect to protected attributes. Our work contributes to the field in three main ways: (1) we construct and release the Multi-EuP dataset, a resource for multilingual retrieval over 24 languages, effectively capturing the multilingual nature of both queries and documents; (2) we explore language bias within the realm of multilingual retrieval, revealing that multilingual IR using BM25 indeed exhibits notable language bias; and (3) we supplement the dataset with rich author metadata to enable research on fairness and demographic bias in IR.[1] [1] The Multi-EuP dataset is available for download from https://github.com/jrnlp/Multi-EuP.

Multi-EuP: Analysis of Bias in Information Retrieval - Abstract and Intro

Recently I wrote an article where I wanted to compare a popular way of creating object instances with DotNet reflection to another. In that article, I put to see which had better performance. The result saw Activator.CreateInstance emerge as winner in one specific case — but another champion emerges: ConstructorInfo. Activator.CreateInstance head-to-head with Type.InvokeMember In this article, I’ll explain how you can get a ConstructorInfo reference using reflection in DotNet. I’ll also expand upon the benchmarks from the previous article, showing you the code and how the results turned out. Understanding ConstructorInfo From DotNet Reflection In DotNet reflection, we get a powerful set of tools for inspecting assemblies, types, and members at runtime. One of the key components of this reflective capability is the class, which belongs to the namespace. This is where all of the goodies are — ConstructorInfo System.Reflection Even the ones shown in this video that can be misused in the wrong hands: https://youtu.be/3Qstc-ND8Oo?embedable=true allows developers to obtain information about the constructors of a class, including their accessibility (public, private, etc.), parameters, and metadata. But one of the best parts, which we’ll be looking at in more detail, is that it enables the instantiation of objects dynamically at runtime without knowing their types at compile time. ConstructorInfo And the best part? We’re going to see that when we compare it to , it’s way faster. these other DotNet reflection mechanisms for making new instances Finding Constructors with Reflection in DotNet – Getting ConstructorInfo In this section, we’ll look at how we can first get instances so that we can leverage them later for object instantiation: ConstructorInfo Get the instances from types ConstructorInfo Find the right constructor … Profit! Something like that, right? Let’s check these code examples out! Get ConstructorInfo for All Public Constructors To retrieve all public constructors of a class, you can use the method without any parameters by using a instance of a particular type. This method returns an array of objects representing each public constructor defined for the class. GetConstructors Type ConstructorInfo Let’s see it in action in this code example: using System;
using System.Reflection;

public class SampleClass
{
    public SampleClass() { }
    public SampleClass(int i) { }
    protected SampleClass(string s) { }
    private SampleClass(int i, string s) { }
}

Type typeInfo = typeof(SampleClass);
ConstructorInfo[] publicConstructors = typeInfo.GetConstructors();

foreach (var constructor in publicConstructors)
{
    Console.WriteLine(constructor.ToString());
} When we’re thinking about dynamically invoking these things, it’s likely going to be the case that we don’t have the reference to the type though. But if we have the name of the type we’re interested in, we can use the following: Type typeOfInterest = Type.GetType("The.Namespace.Of.Your.Type.TheTypeName"); Get ConstructorInfo Including Private and Protected Constructors To get information about all constructors, regardless of their accessibility level, you can use the method with the parameter. This approach allows you to include non-public constructors in the results: GetConstructors BindingFlags ConstructorInfo[] allConstructors = typeInfo.GetConstructors(BindingFlags.Public | BindingFlags.NonPublic | BindingFlags.Instance);

foreach (var constructor in allConstructors)
{
    Console.WriteLine(constructor.ToString());
} Keep in mind that if you watched the video I linked above, this starts to get into the territory of “Should I be doing this?”. Please strongly consider if you need to be accessing non-public things — Someone likely chose that access modifier for a reason. Get ConstructorInfo Matching a Specific Signature If you’re looking for constructors that match a specific parameter signature, you can use (notice that it’s singular). This takes in binding flags like we saw before as well as an array of types that you want to match. GetConstructorInfo Here’s how you could find constructors that take a single parameter: int // Specify the parameter types of the 
// constructor you are looking for
Type[] paramTypes = new Type[] { typeof(int) };

// Use GetConstructor with the appropriate
// BindingFlags and parameter types
var constructor = typeInfo.GetConstructor(
    BindingFlags.Public | BindingFlags.Instance,
    paramTypes);

if (constructor != null)
{
    Console.WriteLine(constructor);
}
else
{
    Console.WriteLine("No matching constructor found!");
} Note that this method will return null when there’s no match, so ensure you got what you were looking for! Creating Object Instances Using ConstructorInfo Once we have a instance, we can start making object instances. This is what we’ll be benchmarking in the upcoming sections! ConstructorInfo In these examples, assume that we already have a instance called constructorInfo. We’d be getting this instance in any of the ways documented earlier in the article: ConstructorInfo object instance = constructorInfo.Invoke(null); The code above shows instantiating an object with a parameterless constructor. We pass in null for the list of arguments that would need to be provided — because there are none. Take note that the type we get back is an . If we have access to the type at compile, we could cast this instance to that type… But if we have access to the instance at compile time there are probably very few good reasons why you would be doing this in the first place. If you don’t believe me, wait until you see the benchmark results. object If we want to instantiate using a constructor that takes parameters it would look like the following: object instance = constructorInfo.Invoke(new object[] { 42 }); This code example shows a constructor with a single integer parameter defined being invoked with 42 as the single integer argument. ConstructorInfo Performance Benchmarks The moment you’ve all been waiting for! You might enjoy watching the video demonstrating these DotNet reflection benchmarks here: https://youtu.be/Djq7eMI_L-4?embedable=true BenchmarkDotNet Setup for Reflection Performance Much like the previous article, I’ve just added a couple of additional scenarios for the scenarios. I wanted to mention that I added TWO scenarios for each class, and that’s because I wanted to demonstrate the performance if you had to go instantiate AND find the back-to-back. I felt like this variation compared to already having the would be interesting to take note of. ConstructorInfo ConstructorInfo ConstructorInfo Here is the full code: //
// This code was written for the following Dev Leader content:
// https://www.devleader.ca/2024/03/14/activator-createinstance-vs-type-invokemember-a-clear-winner/
// https://www.devleader.ca/2024/03/17/constructorinfo-how-to-make-reflection-in-dotnet-faster-for-instantiation/
// https://youtu.be/Djq7eMI_L-4 
//
using BenchmarkDotNet.Attributes;
using BenchmarkDotNet.Running;

using System.Reflection;

BenchmarkRunner.Run(Assembly.GetExecutingAssembly());
//BenchmarkSwitcher.FromAssembly(Assembly.GetExecutingAssembly()).RunAllJoined();

public class ParameterlessClass
{
}

public class ClassicStringParameterClass
{
    private readonly string _value;

    public ClassicStringParameterClass(string value)
    {
        _value = value;
    }
}

public class PrimaryConstructorStringParameterClass(
    string _value)
{
}

[ShortRunJob]
public class ParameterlessClassBenchmarks
{
    private Type? _type;
    private ConstructorInfo? _constructorInfo;

    [GlobalSetup]
    public void GlobalSetup()
    {
        _type = typeof(ParameterlessClass);
        _constructorInfo = _type.GetConstructor(Type.EmptyTypes);
    }

    [Benchmark]
    public void Constructor()
    {
        var instance = new ParameterlessClass();
    }

    [Benchmark(Baseline = true)]
    public void Activator_Create_Instance()
    {
        var instance = Activator.CreateInstance(_type!);
    }

    [Benchmark]
    public void Type_Invoke_Member()
    {
        var instance = _type!.InvokeMember(
            null,
            BindingFlags.CreateInstance,
            null,
            null,
            null);
    }

    [Benchmark]
    public void Constructor_Info_Invoke()
    {
        var instance = _constructorInfo!.Invoke(null);
    }

    [Benchmark]
    public void Find_Constructor_Info_Then_Invoke()
    {
        var constructorInfo = _type.GetConstructor(Type.EmptyTypes);
        var instance = constructorInfo!.Invoke(null);
    }
}

[ShortRunJob]
public class ClassicStringParameterClassBenchmarks
{
    private Type? _type;
    private ConstructorInfo? _constructorInfo;

    [GlobalSetup]
    public void GlobalSetup()
    {
        _type = typeof(ClassicStringParameterClass);
        _constructorInfo = _type.GetConstructor([typeof(string)]);
    }

    [Benchmark]
    public void Constructor()
    {
        var instance = new ClassicStringParameterClass("Hello World!");
    }

    [Benchmark(Baseline = true)]
    public void Activator_Create_Instance()
    {
        var instance = Activator.CreateInstance(
            _type!,
            new[]
            {
                "Hello World!",
            });
    }

    [Benchmark]
    public void Type_Invoke_Member()
    {
        var instance = _type!
            .InvokeMember(
                null,
                BindingFlags.CreateInstance,
                null,
                null,
                new[]
                {
                    "Hello World!",
                });
    }

    [Benchmark]
    public void Constructor_Info_Invoke()
    {
        var instance = _constructorInfo!.Invoke(new[]
        {
            "Hello World!",
        });
    }

    [Benchmark]
    public void Find_Constructor_Info_Then_Invoke()
    {
        var constructorInfo = _type.GetConstructor([typeof(string)]);
        var instance = constructorInfo!.Invoke(new[]
        {
            "Hello World!",
        });
    }
}

[ShortRunJob]
public class PrimaryConstructorStringParameterClassBenchmarks
{
    private Type? _type;
    private ConstructorInfo? _constructorInfo;

    [GlobalSetup]
    public void GlobalSetup()
    {
        _type = typeof(PrimaryConstructorStringParameterClass);
        _constructorInfo = _type.GetConstructor([typeof(string)]);
    }

    [Benchmark]
    public void Constructor()
    {
        var instance = new PrimaryConstructorStringParameterClass("Hello World!");
    }

    [Benchmark(Baseline = true)]
    public void Activator_Create_Instance()
    {
        var instance = Activator.CreateInstance(
            _type!,
            new[]
            {
                "Hello World!",
            });
    }

    [Benchmark]
    public void Type_Invoke_Member()
    {
        var instance = _type!
            .InvokeMember(
                null,
                BindingFlags.CreateInstance,
                null,
                null,
                new[]
                {
                    "Hello World!",
                });
    }

    [Benchmark]
    public void Constructor_Info_Invoke()
    {
        var instance = _constructorInfo!.Invoke(new[]
        {
            "Hello World!",
        });
    }

    [Benchmark]
    public void Find_Constructor_Info_Then_Invoke()
    {
        var constructorInfo = _type.GetConstructor([typeof(string)]);
        var instance = constructorInfo!.Invoke(new[]
        {
            "Hello World!",
        });
    }
} ConstructorInfo Benchmark Results from BenchmarkDotNet The first set of results will be for the parameterless constructor: In the results above, we already knew that without using reflection, we get the best speed. No brainer here. BenchmarkDotNet says it’s so fast it can’t even measure it properly. But we’ll notice that is technically a smidge faster here than using , even if we already had the instance ahead of time. The results are very close, and I have seen this swing the other way. So overall, these two are very comparable in this situation. Activator.CreateInstance ConstructorInfo What happens if we need to use parameters though? The BenchmarkDotNet results above show that a classic style constructor taking in a single string parameter, is an order of magnitude faster than the other DotNet reflection options. Even if we need to look up the instance first, it’s still almost twice as fast as the other options! ConstructorInfo And of course, I wanted to see if primary constructors were any different in behavior: Based on the results above though, they’re right on par! Wrapping Up ConstructorInfo and Reflection in DotNet As we can see from the BenchmarkDotNet results, leveraging can be very performant! Only in the case where we’re dealing with a public parameterless constructor did it seem to be right on par with . Technically in this run it showed that it was a touch slower, but I’ve run these before and seen the opposite case too. Overall, you’ll want to consider if it makes sense for you to leverage this approach for creating object instances — but certainly don’t opt for reflection if you can easily just call ! ConstructorInfo Activator.CreateInstance new() If you found this useful and you’re looking for more learning opportunities, consider and check out my ! subscribing to my free weekly software engineering newsletter free videos on YouTube

Stories, News & Mentions about TYDEI

Tydei’s Stories on HackerNoon