Table of links
2 Is an ethnographic study the right choice?
- The context of your research
- The kind of research questions you want to answer
- What ethnographic studies require from the researcher
3 Planning an ethnographic study
- Finding a site for field work
- Participant or non-participant observation
- Duration of field work
- Space and Location
- Theoretical underpinning
4 Implementing your ethnographic study
- Gaining access and starting up
- Handling your preconceptions
- During the study
- Going Native
- Leaving the field
- Reflective and inductive analysis
- Writing Ethnography for Software Engineering Audiences - Reporting the Results
6 Ethnography and Research Ethics
7 Final comments, Further reading and References
4 Implementing your ethnographic study
Planning can only prepare you so far for the research but here is where the ‘rubber hits the road’. As Fetterman [20] puts it: “The most important element of field work is being there - to observe, to ask seemingly stupid questions, and to write down what is seen and heard.”
4.1 Gaining access and starting up
After reading the previous sections, we expect the reader to have a clear understanding of what ethnography is, what kind of research questions it is best suited for, and the issues to consider when planning an ethnographic study. Furthermore, a “field site” to be studied has to be decided, i.e., a software development team, an open-source community, or organisation. However, as one can imagine, starting an ethnographic study is not only about “showing up” to observe what is going on at the field site. In general, it is important to have a key informant (also called a ‘gatekeeper’) that can facilitate the researcher’s access to the informants. This can be a project manager, team lead or a scrum master. This key informant is a person who is trusted, and possibly respected, by the informants and is willing to introduce the researcher to them so that they can start the data collection. In summary, this key informant negotiates access with the informants before the researcher goes to the field. Often, a good idea is to let this key informant introduce the researcher and suggest one or two initial members that are willing to be observed or interviewed. By interacting with these initial members, the researcher will have a chance to meet other team members and can negotiate with them additional opportunities for data collection.
The key informant is also the person that the researcher will look for to ask clarification questions regarding the project being studied, its context, the diverse roles being played by the informants, the software development organisation, and the software being developed. This information often will also indicate documents and tools that the researcher should be aware of, and if necessary, the key informant will make the necessary arrangements so that the researcher can access these documents and/or tools. The key informant can be thought of as a facilitator who will allow the researcher to conduct the ethnographic study.
Part of starting up the actual field work is also to agree on the legal side of the collaboration and develop and sign cooperation agreements containing nondisclosure agreements and agreements about the intellectual property that might be developed as part of the research project. As part of starting up the field work, the researcher also needs to consider the research ethics relevant for this project and decide, maybe together with the key informant, on how to handle the informed consent. The latter topics are discussed further in section.
• Don’t impose your point of view. Let your student express their perspectives • Challenge your student’s observations but gently • Spend some time focusing on the use of language. How informants talk about their work or aspects of their organisation is very informative but attention to this kind of detail is not often emphasised in software engineering classes.
4.2 Handling your preconceptions
One of the challenges of ethnographic research for a software engineer studying software engineering practices is that the researcher brings an education to the field that states how software should be developed. For example, having attended a software engineering course, the field worker might expect a ‘stand-up’ meeting to take place in situ by people actually standing up. However, in virtual software development, a stand-up meeting might mean that people sit in front of the screen communicating using a virtual meeting environment [16]. Similarly, the textbook version of pair programming describes a “driver” and “navigator” role, but in practice pairing rarely adopts these roles formally, as the intensity of development results in a much more fluid exchange of activities [45]. So how can a researcher handle such a situation, where the observed team is not doing things ‘by the book’? Ethnographic research talks about ‘bracketing’ prior assumptions and knowledge. In the above example it would mean accepting that pairing in practice changes shape when the focus is on developing code rather than following a pre-defined procedure, and that stand-up meetings may look different for different teams depending on their context. The research can then focus on how development is implemented in this context, what is the rationale for deviating from the textbook way of doing things, and how have these practices developed.
4.3 During the study
Observation is key to ethnographic studies and both participant and non-participant observation are legitimate forms of ethnography [21]. The data collected for observation is largely in the form of notes. Field notes are an important aspect of the work done by the ethnographer and the researcher must be prepared to take notes during the study. The question, though, is what notes to take? On the one hand, the researcher can document as many things as possible including the events, things people say, interruptions, event participants, documents being accessed or discussed, because at the start of the study it’s hard to know what will turn out to be significant. However, documenting everything will quickly become overwhelming. Textbooks sometimes recommend a template or schema for field notes. Such a schema can be very helpful and provide useful scaffolding at the start. Filling in the schema should not become the main purpose, though, and a tendency to just ‘tick the boxes’ needs to be resisted. The main purpose is, of course, to make sense of the team’s work practices and to document what is relevant to that aim. An alternative to schemas is to use a framework of questions. There are simple frameworks such as “who is there”, “where are they?” “what are they doing”, and more detailed ones such as the following based on Robson and McCarten [47] (p. 328):
• Space: What is the physical space like, and how is it laid out?
• Actors: What are the names and relevant details of the people involved?
• Activities: What are the actors doing, and why?
• Objects: What physical objects are present, such as story cards?
• Acts: What are specific individual actions?
• Events: Is what you observe part of a special event?
• Time: What is the sequence of events?
• Goals: What are the actors trying to accomplish?
• Feelings: What is the mood of the group and of individuals?
These are only intended to help keep a focus on relevant issues so that the ethnographer is not overwhelmed, and will evolve over time. Today, many researchers use computers for their field notes, which makes it easier to link to other field material such as (sample) documents, photos, and recordings. But handwritten notes are also popular and some find them a better way to engage with their surroundings. Each researcher will need to find their own preference. However, as mentioned in sections 3 and 4, the focus of the study will evolve and the field notes will reflect that. For instance, after noticing that the software architecture influenced the interactions between the software teams, de Souza and Redmiles [10] started to pay more attention to the inter-team interactions because they were sources of tension between the different product teams. As the focus changes, ethnographers also adopt different data collection techniques, for instance, during the field work the researcher can decide to follow an artefact, e.g. a pull request, bug report, a presentation or the documentation of a piece of the software architecture. When this happens, the researcher should be prepared to ask for access to these artefacts which may mean a physical (print out) or digital copy of it. By following an artefact, the researcher might shine light on a different set of new events, activities and/or informants who often will bring new perspectives into the field site being studied.
Last but not least, it is important to note that field notes will contain both objective information such as quotes, timestamps of events, links to recordings and photos, and subjective information, such as the researcher’s reflections and interpretations of what is going on. Objective information is important, to anchor your emerging understanding of the work practices in actual observations. Verbatim quotations are extremely useful to present a credible report of the research. According to Fetterman [20], “quotations allow the reader to judge the quality of the [ethnographic] work - how close the ethnographer is to the thoughts of natives in the field - and to assess whether the ethnographer used such data appropriately to support the conclusions.”
The subjective informationincludes the researchers’ tentative interpretation of the events being observed, “hypotheses” about what is being observed, and questions the researcher asks themselves when collecting data. These reflections can evolve into important memos that in turn will develop into the parallel analysis of the data. It is, though, essential that the researcher clearly differentiates the subjective from the objective information in their field notes. This can be done by using different colours in a paper based field diary, or by splitting the page vertically or horizontally. Reflection and emerging hypothetical interpretations are, in qualitative research validated in the field. For instance, during data collection De Souza and Redmiles [57] started to wonder whether the phenomena observed would occur only with novice team members. To “validate” whether this is the case, they decided to observe and interview senior team members. Through this they found out that the phenomena was more general and occurred with team members of different experience.
In some cases, the events being observed will unfold so fast that the researcher will not be able to take notes. In this case, ethnographers allocate time to record their memories in their field notes as soon as possible after the event, and by the end of the day at the latest. Nowadays, recording meetings is common and straightforward since some participants are online. In this case, it is important for the researcher to negotiate access to these recordings because they will be important data points for data analysis. They should be linked to the field diary so that the researcher later can connect them to the context in which they took place. However, being able to record meetings so easily is a potential disadvantage because it’s easy to collect a huge amount of data, and there is a tendency for the researcher to lose concentration if they know data is being captured through other means.
In addition to observation, conducting ethnographic studies will likely employ a range of data collection methods including interviews and document analysis. Interviews are a chance to clarify meanings and the history of what has been observed. Such interviews might be formal or informal. Informal, or impromptu, interviews are often in situ and opportunistic; they will often occur in the context of an observation. In other words, the ethnographer, after witnessing an event that seems interesting, surprising, or even expected, will use the opportunity to engage in a quick conversation with the informant to find out more details. These informal interviews will occur when opportunities arise, for instance, during a break or while waiting for a meeting. All this is done having in mind that the role of an ethnographer is to understand the member’s point of view. Formal interviews require an agenda and a negotiated date and time, but may be the only opportunity to speak with some informants or to explore some kinds of issue.
In our experience, a good ethnographer strikes a balance between conducting formal and informal interviews. What this balance looks like depends on the context and the flexibility of the informants’ working day. Too many informal interviews with the same informants can become irritating. Similarly, conducting formal interviews means asking people to allocate time for the conversation. However, checking information and making sure that observations and reflections are well-founded are fundamental. A good ethnographer also needs to pace themselves, plan their days carefully and be systematic about keeping records. This is particularly significant because data collection will go on for several days or even weeks, and it is easy for memories to fade or become confused. The researcher needs to be constantly reflecting about what data is being collected, including around events, conversations, and artefacts, and revisit the emerging analysis concepts in light of the new observations. Ethnographers need to remember to avoid judging the work being observed and to keep focus on the informants’ point of view, i.e., why they do the things they do. Being careful about data collection also means collecting data from different informants to gather different perspectives while, at the same time, not being a nuisance.
Finally, it is important to emphasise that ethnographic research, similar to other qualitative research methods, requires cycles of data collection and analysis. In other words, ethnographic research requires negotiating continuous and evolving access to the site. As discussed in the next section, during the data analysis new foci might emerge and require new directions for data collection, such as access to new informants or new events or new documents. So the ethnographer must be continuously negotiating access to these during the research process.
• Being judgmental about observed practice SO focus on why it is the way it is.
• “Going native” meaning that the way informants behave and achieve things become ‘normal’ SO talk to other researchers and hold debriefings to help you regain the questioning mindset.
• Losing your “strangeness” perspective and not knowing what needs to be told for others to understand your interpretations SO try explaining what is happening to an alien.
• Human tendency is to gravitate towards people you like but in the field it’s important to attend to all informants SO be aware of your own preferences and biases.
• It is easy to focus on a small number of informants because you become comfortable with them SO make sure you choose a range of informants to get different perspectives, i.e., novices and experts, male and female, informants with different roles, etc.
• Observation can result in lots of data, some of which is not relevant SO focus on what to pay attention to and choose carefully what data to collect.
• Assumptions or pre-conceived ideas can lead to misunderstandings SO challenge your own assumptions and ask dumb questions to establish that your understanding is correct.
• Recording meetings and interviews is straightforward and easy but can result in huge amounts of data and loss of concentration SO keep taking field notes even when the interactions are being recorded.
4.5 Leaving the field
Eventually, the researcher will have to leave the host organisation or the open source community, because the software development project finishes, the research project ends, or because the researcher has learned what could be learned from the ethnographic field work. Leaving the field should be done in a way that allows the researcher to come back to ask clarifying questions and open up for future collaboration on new mutually interesting research questions. It is therefore important that the researcher informs all of the informants that they are leaving the site and what will happen next with their data and its analysis. Saying farewell to the team can be combined with a presentation of (preliminary) findings and insights that can also be used as an occasion for member checking. Member checking is an important aspect of any qualitative research (see also Section 5). It involves presenting the results of the research to the informants so that they can “validate” them. In ethnographic research, one way to do this is to formally present the results to the key informants who facilitated the ethnographer’s access (see Section 4.1), or even to the whole development team. Another way is through informal presentations during the final days of data collection, such as during a break, while waiting for a meeting, etc. The format in which this is done varies depending on the team, project, and overall context. On the one hand, feedback from the participants is important for the trustworthiness of the findings. On the other hand, presenting the findings can also serve as a ‘thank you’ for the team and can support the reflection and improvement of their own practices. If an organisation or team is interested enough in having a researcher on site then they will also be interested to know what insights have been found. As one of our collaborators once said “the problem with running a project is that you’re running, but having you here allowed us to stop and reflect”.
Authors:
This paper is