Formative Research

Due: Thursday, Feb 25 11:59PM
Point Total: ~15% of your TA grade (6.8% overall)

Assignment Overview

Your assignment is to extend and refine the existing “elevator pitch” for your project into a longer, more comprehensive project proposal based on formative user research and competitive landscape analysis. The maximum page length is 5. Note: images, figures, and tables are free and do not contribute to the page count. The appendix (required) and reference list (also required) do not contribute to the page count either. You will work on this assignment with your project team.

A Brief Timeline
  • On Thursday, February 11th, we will conduct in-class brainstorms. Specifically, we will break out into our teams and brainstorm/iterate on the original project pitches with a particular focus on who are the target users, how can interactive technology play a role in solving our proposed problem, and what particular aspects are you, as a team, going to solve this semester.

  • On Thursday, February 18, you will come to class with a draft of your proposal and an action plan for completing the rest of the assignment tasks. You will get feedback on your draft and action plan from at least one other team. One person on each team will be a scribe. Please include a scan/picture of your notes in your final report.

  • On Thursday, February 25, you will submit your final PDF.

What to Do

  1. Brainstorm. Ideate. Generate as many ideas as you can about the trajectory of your project this semester. Brainstorm together. Follow Kelley's recommendations for a "perfect brainstorm" (source link) as well as the strategies we discussed in class. You will submit your brainstorm notes (e.g., scans/sketches of your notebook, pictures of whiteboards) with your report.

  2. Perform a competitive analysis and build an inspiration library. This is extremely important and can be weaved into your brainstorming sessions. You must understand how your ideas are related to other similar technologies/applications that are out there. Your report must include an analysis of at least eight specific commercial products or academic projects along with statements about expected differentiation. Include screenshots whenever possible (this is part of building your inspiration library--remember, creation through curation). Use these investigations to fuel your brainstorms and remix!

  3. Identify your primary target user groups. These groups should be justified in your report. For example, if you are making a new on-campus map application, you might identify four classes of users: (i) tech savvy students, faculty, and staff who are already familiar with the campus; (ii) one-time visitors to campus with SmartPhones; (iii) new students, faculty, staff trying to learn the campus and the buildings necessary to their education/work; (iv) people with impairments (e.g., persons with low-vision or mobility impairments).

  4. Narrow down on one or two classes of target users. In this class, we don't have time to design for all primary users groups, so just focus on one or two. Your report must include why you made the selections that you did.

  5. Select two formative user research methods and conduct your formative research using these methods. One of these methods must interface directly with your target users (e.g., either via interviews, surveys, observation, etc.). Again, as with all design decisions in this class, you must justify which methods you selected. For guidance, see the IDEO Method Cards (source link; note, however that not all IDEO cards describe formative user research methods--some of them are design methods), An Ethnographic Approach to Design by Blomberg and Burrell (source link), Contextual Design by Beyer and Holtzblatt (source link), and the lecture slides. Record your notes, interview responses, survey results, etc. You must include these in your report (in the Appendix).

  6. Analyze your collected data and synthesize design implications. What did you learn? How might this affect your design?

  7. Write the report!

The Report

You should use a 10- or 11-point serif font for the prose typeface. You should choose and be consistent with heading fonts. Images should be placed inline in the text rather than the appendix. The document layout should be single column.

Section 1: Title and Abstract

The title can be an updated form of the title from the “elevator pitch” and the abstract will be a refined version of your “elevator pitch” based on your new brainstorming/ideation sessions. Note, however, that the abstract must be no longer than six sentences. Roughly, your abstract should contain a motivating sentence about the problem and why it’s important, a sentence on past solutions along with their limitations, a sentence on your proposed solution and how it is unique from past approaches, a sentence on how you plan to evaluate the effectiveness of your proposed solution, and a sentence on who this will benefit and why. Your abstract must address each of the aforementioned points.

Section 2: Introduction

The introduction should clearly articulate the problem and why it is important. It should also include specific high-level goals for the project (i.e., what does a “win” look like for this project and what metrics are you going to use to know that you got there). You should be sure to describe how your proposed solution is novel or, at the very least, how it may extend existing solutions. The introduction should be two to four paragraphs. If your project has changed substantially from the original pitch because of your competitive analysis and formative user research, then one of these paragraphs should explain these changes.

Section 3: Background / Review of Past Work

This section should provide a background context on the problem (e.g., its history) as well as a review of past work (i.e., competitive landscape analysis). I expect at least eight specific references to either related commercial products or research papers. When relevant, you should include a figure of the related application, tool, product, or webpage with a numbered figure caption below. When referring to past work it’s important to include a description of the past work, why it’s relevant and significant to the proposal at hand, and how your proposed solution is different. When citing work, please refer to the References section below.

Note: If you are having trouble with this, post a question to Canvas or email me or Matt. It’s absolutely essential that you understand the current solutions within your problem space so that you (1) don’t reinvent the wheel; (2) can fully differentiate your solution; (3) can extend and build upon good ideas from past work and avoid the bad ideas from past work.

Section 4: Target Users

In this section, you should describe your list of primary target users as well as your narrowed list of the one or two you selected for this semester's project. Your should enumerate their needs, and why your proposed solution benefits them. Most, if not all, of the project proposals should have more than one user group. Don’t believe me? Think about the IDEO shopping cart redesign. You may think that “shoppers” are the only users of the carts, but we know better. Shoppers may be the primary users—perhaps the most important users—but others include the employees who must clean and collect the carts from the parking lot, the mechanics who must fix the carts, and even the homeless who might want to use the cart for their own reasons. All of these user groups (with the exception of the last group) must be able to effectively use the cart or the cart design fails. Even further, the primary user group—shoppers—can be further categorized into subgroups: power shoppers, shoppers with children, shoppers with disabilities, shoppers in a hurry, very tall shoppers, etc.

Section 5: Formative User Research

In this section, you will introduce and describe your two formative user research methods. For those methods that interface directly with your target user group (whether it was via observation, interviews, surveys, or some other method), you must have the following subsections (recall that you were to select at least one method that interfaced directly with your target user group).
  • Intro to method and why you selected it. This should be one paragraph and provides a nice overview of the method you employed and why you selected it.

  • Participants: This should be one or two paragraphs describing your participants (e.g., their demographics) and how they were recruited. You should also justify why these participants were selected and how they relate to your project. Please include a participant table where each row is a participant and the columns describe attributes of that participant related to your project (e.g., gender, age, etc.). Click here for an example from my dissertation (right click and open image directly to expand):

  • Procedure: This should be one paragraph and describes how you executed the formative research (e.g., where did you conduct the interview, how long did it take, what are some example questions you asked)

  • Data and Analytical Method: This should be 1-2 paragraphs and describes the data you collected and how you analyzed this data. For example, for data collection did you record the interviews, did you take notes, etc. For the analytical method, how did you synthesize the data? Did you discuss it as a team and organize your notes into higher level themes/concepts?

If both of your formative user research methods involved users, repeat the above structure for both. If not--for example, you used Informance or Role Playing (see IDEO Method Cards, source link)--then your structure should be: (i) Intro; (ii) Procedure; (iii) and Data and Analytical Method. Adapt them accordingly.

Section 6: Formative User Research Results

This section should have two subsection headings: Results from <Formative Research Method 1> and Results from <Formative Research Method 2>. Organize your results into themes/categories. Emphasize why they are relevant to your project and discuss their significance.

Section 7: Conclusion

Conclude with one or two paragraphs on your main findings and your planned work going forward.

Section 8: References

A list of references formatted according to the ACM Style guide (link), Chicago Manual of Style guide (link), or the APA Style guide (link). I use Mendeley for this (works great)--others use Zotero , Papers, or EndNote. You can also keep them organized by hand (I used to do that for a very long time). :) And remember, you must include at least eight related work references--you are welcome to have more. You can also use references in other places in the report (e.g., if you utilized a specific method, you can cite that method from the literature).

Section 9: Appendix

The appendix should include:
  • A copy of the original elevator pitch (in its unmodified form) followed by a brief paragraph describing how the proposal has evolved since its initial composition.
  • Your brainstorming notes (e.g., scans of your notebook, pictures of whiteboards, pictures of zaniness).
  • Notes from the in-class critique session on Thursday, February 18
  • Your raw data collected from your formative research (e.g., notes, interview responses, survey results)

Additional Requirements

You must include several visual elements that complement your submission; again, these elements will not count towards your total page count. Images, graphs, and tables should all be featured with captions that explain them. Please inline these visual elements into the report itself rather than placing them in the Appendix (I like seeing the figure/table close to the associated text). Remember to cite any sources used in your references. Please email me or post to Canvas if there are questions or concerns about this requirement.

Examples

Here are a few high-quality examples from previous semesters. Note that we change our assignments each year (from small changes to significant overhauls), so please follow the instructions in this assignment and the rubric closely.

Submission

You will submit two PDFs to Canvas.
  1. PDF One: The full report
  2. PDF Two: The full report with all images, tables, etc. removed (and the pages repaginated accordingly). This will be used to assess the 5 page limit. We will deduct 20% for each page over the limit (prorated for partial pages). Note: there can be no other differences between PDF One and Two.

Rubric

You will be graded on how well you execute on each of the above seven sections. You will be additionally graded on creativity and writing quality across the entire proposal. Please follow the directions carefully. In the past, I've had students completely miss sections in their report. Finally, I will be factoring in peer review to help assess the quality of the work and the output from each team member.

You can download the rubric as an Excel file below:


We've also taken a screenshot of the rubric for your convenience. You can click on the image below to enlarge.
TA02-FormativeResearch-GradingRubric_v1_MLM_cropped_650w.png

Common Mistakes

  • Including figures but without captions: Each figure should have a caption that looks like: "Figure X: <caption text>" where X is the figure number.

  • Not referencing figures in prose: Each figure must be referenced in your prose. If you have difficulty figuring out how to refer to the figure in your text, it's likely that the figure is tangential/irrelevant and can be removed.

  • Enumerating related work but not differentiating: A very common mistake--even with graduate students--is to simply enumerate work related to your proposal rather than explicitly describing how your proposed project is similar/different. The "Background / Review of Past Work" section should be seen as a form of rhetoric (of argument) articulating how your proposed solution is informed by past work but sufficiently different from it. You should never simply list a product or past piece of research without differentiating your solution.

  • Enumerating related work but not synthesizing: An acceptable "Background / Review of Past Work" simply lists and differentiates past work and directly links it to the proposal at hand (by drawing similarities/differences). A better approach is to offer high level synthesis and categorization of past work. Think about the higher level relationships between previous work and how they fit together. Use this to provide sub-sections in this section (e.g., related to specific features, techniques, etc.).

  • Not justifying recruitment method. It's not enough to simply tell me how you recruited participants for your formative studies, you must also provide a rationale (that is, why did you choose that recruitment method and how do you think it may affect your findings).