A study on how expert cartographers use software to create maps
Create personas for cartographers to better understand the users of ESRI's GIS desktop product, ArcGIS Pro
I interviewed 12 expert cartographers at ESRI and analyzed the interview transcripts through open qualitative coding
I pursued this project during my summer internship at ESRI from June to September 2018
This was an individual research project
I interviewed 12 expert cartographers at ESRI and analyzed the interview transcripts through open qualitative coding. Out of the 12 cartographers, 7 created maps to identify patterns and implications within complex data, and 5 made innovative visualizations for maps.
I created 2 personas, one was more research specific and the other was more design oriented within the domain of cartography.
Besides the personas, I wrote a report that described the goals, problems, motivations, common workflows, and skill sets of the cartographers in much more detail. I also covered problems cartographers faced with ArcGIS Pro and provided recommendations for the same.
Due to a signed NDA, I will not be able to show the personas here, instead I will describe the process followed to create the personas and reflections on this process
These personas and report were created for:
- Designers to understand and empathize with their users
- Product Owners to know their user base
- Upper management to understand the importance of persona based research
I first decided the outline of the persona, to make sure that the deliverables could be useful for all stakeholders: goals, skills, motivations, pain points, workflows, perception of the software. This outline guided the interview questions that I would ask :
- Warm up questions in which they talked about their role, responsibilities, what cartography meant to them, and what they enjoyed about it.
- Questions about specific projects and their workflow, to help me understand the process they go through while making a map.
- Cartography specific questions such as challenges, time consuming processes, important processes
- The technology they use to make these maps, their likes, dislikes, and problems they face.
I conducted 2 pilot interviews and polished my questions.
Availability and access to participants who I could easily interview, guided the decision to recruit cartographers working at ESRI. Hence, I limited the scope of the study to expert cartographers in order to match the profile of the people I was interviewing. I recruited participants through email. I Interviewed both researchers and designers in cartography, experienced and relatively new people. I conducted 12 interviews for around 2 weeks, that lasted an hour each. I took notes on my laptop and recorded the audio, and then later transcribe the audio to fill in any gaps I missed during the interview.
I conducted an open qualitative coding session on the transcripts. I created sub codes called goals, skills pain points, etc. The first round was about finding patterns in the data about what people were saying and assigning sub codes too them. In the second round of coding, I churned out the patterns in more detail. I gave a structure to all the codes, and categorized them into themes. I then took these themes and codes and put them on a white board, to help me think visually, and make any connections that I might have missed. This white board activity also helped me think about design recommendations.
Due to my limited knowledge in cartography, I asked 3 participants and my colleague to go through my report and personas and provide their feedback. This proved to be a useful activity as I corrected a few things in the report based on their suggestions.
- Different personas emerged: Half way through the interviews I realized that there were 2 types of cartographers – research and design oriented. I realized that most of my participants fell into the research category, and had to change my recruitment strategy. I asked my participants to refer people who did more design work and I set up more interviews with them.
- The 'common' user: The data collected represented only expert cartographers, and was not representative of the ‘common’ user. We hope this can be a starting point for more perosna based research, and the next steps would be to interview novice and other common users.
- Building domain knowledge: I did not have any domain knowledge about cartography and I was more comfortable after the first 3-4 interviews. I kept asking participants about what things meant, and asked them to show me what they meant on their computers. There was one participant who told me about a lot of things in cartography, which helped me understand concepts and made me more comfortable for the next interviews.
- Speed in industry based research: I realized that Identifying what the stakeholders want out of these deliverables is important. My coding approach was very academic and in-depth and in hindsight, that was not required. For example, I attempted to relate the problems faced by cartographers in the software back to the workflow , hoping to understand how and where the software impacts them. However, stakeholders only cared about the problems identified and recommendations on how to fix them. IDentifying the stakeholders' needs could have helped me work through the analysis much faster and more efficiently.
- Stakeholder Needs: Should have interviewed stakeholders to understand their needs out of this project
- 'Actual' user: Should have talked to the participants on their views of people who would represent the ‘actual’ user using ArcGIS product
- Pilot: Have at least 3 pilot interviews! Even if they are not with the user base. It helps polish the questions
- Interview questions: Re-frame the questions such that they are more casual, direct, and make people want to tell their story
- Taking Interviews: Typing during the interview worked for me. It helped slow down things, gave me time to think over what they said and how to probe further. It also made the transcribing process much faster. Taking hand written notes can work better as you won't lose eye contact.
- Recognize your limit: Take no more than 3 hours of interview a day! Recognize your limit. It is important to show energy since that sets the mood of the conversation.
- Re-visiting interview notes: Revisit the interview notes the same day. Even better, just transcribe the interview and study it so you are better prepared for your next interview. Don’t start coding though, that can bias your next interview.
- Efficient coding: chunk by chunk coding of the transcripts rather than a line by line approach. See scope of the project, what does the stakeholder want? With projects that have crisp goals and short deadlines, it is okay not to do a detailed line by line coding.
- White-board visualizations: Although the White-board visualization helped in making connections, it may not be required for closed ended studies. They are important to find connections between themes which is more fruitful in open ended studies
After reflecting on how I changed the interview questions after the pilot study with my colleague's help, I realized that questions should be casual, direct, and encourage people to tell their story.
- Warm-up questions: In warm up questions, the interviewer should get the interviewee talking about themselves. Instead of asking ‘What is your professional background’, ask ‘How did you get here? What was your journey?' Direct the questions towards them, make them more casual. Think of how to frame this question such that people would answer it like telling a story
- Making assumptions :I assumed participants would talk about cartography, when asked about their work, but that didn’t happen as people did a variety of work. Be direct, and don’t make such assumptions. I added cartography specific questions later.
- Ask direct questions : Avoid vague questions that you cannot answer yourself: what are your goals and how do you define progress? What are your responsibilities? These questions are too vague. Instead, ask ‘what does a cartographer do? what do you enjoy the most about cartography, how did you get interested?’ All these questions are very much directed to the person.
- Remember the aim of the study: Go back to what you want out of the interviews and make your questions. I wanted to know how they made maps, so I asked about specific projects: the audience, process, and challenges
- Easy to answer: Make questions more direct and easy to answer. For example, instead of asking ‘What tools cause hindrances to your work?’ changes to ‘what tools don’t you like, or are difficult to use and why?’. This shifts the focus to the tool rather than hinderance in workflow
- Typical questions: These are typical questions that should be asked while evaluating a product: what you like, what you dislike, what would you change, favorites, frustrations, frequent tasks?
- Better interview questions: I had time to analyze what was wrong with my questions and how they were re-worded. I realized my questions were very formal, and I should ask questions in a way that make people want to talk. I also understood that with studies that have a clear goal, it is okay to ask very direct and specific questions. I think it is important to conduct at least 2-3 pilot studies, as it helps polish the questions before the actual interviews.
- Taking better interviews: It is really important to stay alert, and I know now that my limit is 3 hours of interviewing a day.
- Cartography:I learnt a lot about cartography, a field that I knew absolutely nothing about, and I am fascinated by the talent and skill-set these people have.
- Coding efficiently :I learnt how to code efficiently, for studies that are not very open ended, it makes more sense to code bigger chunks rather than a line by line approach.
- Business vs Academic research: I realized how to take different approaches based on the time crunch, resources available, and needs of the stakeholders.