r8 - 28 Feb 2008 - 19:39:13 - MimiYinYou are here: OSAF >  Product Web  > OpenDesignProcess
Open Design, Not by Committee We are giving a talk at OSCON 2007 on our design process: OSCON2007.ppt

Open Design Process

Today, our process is firmly rooted in user-centered design and for the past year, we have been experimenting with different ways to open up that process to OSAF staff and our fledgling developer and user community.

What do we mean by 'User-centered Design'? The short answer: We start with the user and we always go back to the user.

Chandler's design is an amalgam of user workflow analysis (original and borrowed) and inspired feature ideas. Yet, for a long time, the project was rich in vision and poor in focus and direction. Over the past two years, we have dramatically narrowed the scope by identifying a target user and a target user group: A knowledge worker managing a small group or household; and defining a problem space: Keeping a small group of people who work closely together on the same page, aware of what's going on and able to communicate and collaborate with one another with minimal effort. Additionally, part and parcel in solving this larger collaboration problem is also figuring out how to help individuals keep track of their personal information.

Defining target users and problem spaces involves live user interviews, keeping on top of relevant research and looking at the products potential Chandler users use today. All of these inputs help us sort out how people like work, how people don't like to work and most importantly, how people work with one another. By surveying a wide variety of people, we are also able to identify patterns in roles and responsibilities, which is crucial to identifying prototypical target users and building convincing personas for them. By collecting real usage scenarios and looking for patterns across users and use cases, we are able to model 'true-to-life' workflows that take any given user task out of the Chandler vacuum and places it in the context of real-world usage, as in: On a specific person's desktop; Amidst many other applications; Across several 'usage sessions'.

From these workflows emerge a coherent story about information architecture and the user's conception of Chandler's laws of physics. Finally, from these workflows emerge a list of features and functionality. This is what we mean by 'Starting with the user'. We 'Return to the user' when we get into trouble as well as for periodic check-ins, just to make sure we haven't strayed too far down the path of speculation. This means that user interviews are on-going, design issues are consistently redirected from abstract speculations back to concrete, specific use cases, lightweight usability tests are conducted, though not as often as we'd like. For the past year, we've been using Chandler internally along with a few dedicated users. The feedback we've accumulated during this period is our starting point for 1.0 design and planing.

The question remains: How does user-centered design work in an open source project?

User-centered design is not new. What is new is user-centered design in an open source setting. To start off on a positive note, there are many things designers can learn from open source and there are many opportunities to find common ground. The list (and therefore verbal) - oriented culture of most open source projects dovetails nicely with how user research, workflow analysis and usability testing are conducted. (Which also means it doesn't work so well with the graphical end of design work.) Negotiating between open discussion and collaborative problem-solving with a coherent architecture and project direction is a balancing act well-known to designers.

Nevertheless, it takes a serious optimist to believe that user-centered design and open source software development can play well together. A gross over-simplification of some of the "challenges" we face is available in the table below.

The Engineering Perspective The Design Perspective
Organized around functionality Organized around workflows and scenarios
Simple to understand how it works Simple to use given your daily needs
Consistent in terms of technical semantics Consistent in terms of human semantics
Let people design their own way of doing things Not everybody wants to build their own system
Used to clearly defined domains Hard to modularize. Can’t run a functional test on design proposal

Both engineers and designers can be committed to building 'good and simple user interfaces'. Yet at times, our diametrically opposed 'starting points' result in miscommunication and conflict. But any foray into collaborative design will be beset with misunderstanding and disagreement because ultimately what makes for a 'good and simple user interface' is subjective.

So why are we nevertheless committed to having an Open Design Process?

What we hope to gain…

  • Brainstorming: More brains, more ideas.
  • Validation: More people, more use cases, more perspectives.
  • Quality assurance: Everybody’s a tester.
  • Feedback: Get users involved in giving us feedback.
  • Community: More engagement, more usage!

What we don’t want to lose…

  • End-user focus: How do we avoid simply listening to the loudest person on the list.
  • Coherence
  • Speed in making decisions and iterating on designs.

What do we mean by 'Open Source Design'? A snapshot of the last few years of design at OSAF.

At the start of the project, there were reservations about opening up the design process, even internally to OSAF staff. Since then, we have found several natural crossover points and for the past 3 years have been navigating towards broader and richer participation in design.

The tools we use . A bug-tracking system for logging problems and finalized design proposals, a mailing list for discussing design issues and a wiki for exploring designs, defining feature requirements and specifications. We conduct the majority of our design discussions on the list. Off-list meetings are summarized on the list both for the sake of transparency as well as to give others an opportunity to participate.

Collaboration has been most successful at stages in the design where what's most important is to collect a broad array of inputs. We put out a call for personal anecdotes, use cases and real life examples. We essentially conduct a large-scale brainstorming exercise where the motto is: The more toppings the better. However, not everyone participates in every design discussion. If that happened, no one would have any time to write code. For any particular issue, there usually exists a self-selected smaller group of anywhere from 2-6 people who are invested in the issue. The group may include the developer(s) implementing the feature, a designer, a product manager and if we're lucky, the user who encountered a design problem. Even in these smaller group discussions however, our tried and true technique is to focus the discussion around specific, concrete use cases. Oftentimes, in the absence of a user, we appeal to the wider community to come up with examples from their personal store of experiences.

However, the most fruitful and in many ways, most critical area of open collaboration is in our design discussions with engineering. Unlike many software processes where program or product managers oftentimes write specifications detailing not only what features and functionality are needed, but how they should be implemented, we generally arrive with relatively lose workflow storyboards and mock-ups, intended to function less as prescriptive specs and more as visual aids to make our ideas more concrete. In other words, the 'design' is just the beginning of a very long road towards a tangible implementation. More often than not, the design changes dramatically as implementation discussions expose unintended side effects of the proposed design and/or deeper discussion simply reverses design assumptions and we are sent back to the drawing board to resolve newly unearthed issues.

Some high-level rules of thumb we've developed include:

  • Feedback, input and ideas are fed through a moderated design process to ensure that we get the best of collaboration and avoid the pitfalls of design by committee.

  • Consensus is not required to make decisions. Agreement is desired, but when disagreement persists, the decision driver is there to make the call.

  • Voting is used as a gauge, a way to collect input; not as a way to make decisions.

  • Ideas aren’t simply accepted wholesale. They are fodder, to be fed through the design process. Feature ideas are always taken back to motivating use cases.

Much of this is common sense. Applying these techniques in the midst of a long 'into-the-weeds' discussion is trickier.

Open Design in Practice

Our purported goal is have an effective collaborative design process, a process that thrives on the collective wisdom of the community yet avoids the pitfalls of design by committee. This is easier said than done.

We've discovered that Open Design is 1 part listening and 2 parts demonstrating that you have listened and making transparent decision processes that are usually closed. This responsibility lies squarely on the shoulders of the person(s) driving the design discussion who also happens to be the person who gets to make the final call. In other words, with great power, comes even greater responsibility wink Or framed yet another way: Shouting match doth not an open design process make.

Some questions we've had to ask and answer include:

Q What does it mean to be a Design Driver?
A Taking responsibility for the ultimate 'success' of the design discussion where 'success' is defined as 'driving the discussion to a resolution key stakeholders can live with such that the project can continue to make forward progress'.

Q What does it take to drive a discussion?
A As any denizen of collaborative processes can tell you, guiding discussions is a bit of a black art. Here is our best effort thus far at articulating what is essentially a 'feel your way through it' process.

  • Identifying the question at hand.
  • Identifying key stakeholders.
  • Gathering inputs; and
  • Elicit helpful feedback;
  • Identifying what the issues are; and
  • Focusing the discussion around addressing and solving those issues, as opposed to just continually coming up with more things to complain about; and
  • Driving it to a satisfactory conclusion that results in concrete next actions.

It also helps to:

1. Clarify expectations:

  • What are we trying to get out of the collaboration?

  • What do we expect out of the people who want to collaborate? How much context do people need in order to engage effectively and productively on any given issue?

  • How are we going to use the feedback people give us? Summarizing feedback is good. But being able to tell people up-front, what the point of the exercise is, is even better.

  • Summarizing inputs, feedback and discussion results.

2. Define goals: What are we trying to accomplish? What problem are we trying to solve?

  • Articulating and acknowledging people's concerns, so they know they've been heard and understood and that their issues have been considered in the decision-making process.

  • Extracting design requirements from people's concerns.

  • Tying requirements back to the goals: How does this requirement help us reach our goal?

  • Figuring out what makes something a legitimate requirement: Is this truly a requirement? Can we reach our goals without meeting this requirement?

  • Explaining design requirements that people may not understand right away.

  • Continually re-focusing the discussion around: How do we reach our goals? How do we meet requirements? How do we address concerns? How do we bridge the gaps where people disagree?

  • Turning requirements into criteria for decision making. I think that means prioritizing requirements, evaluating solutions in the light of goals and requirements, etc.

  • Tying the ultimate decision back to the criteria.

The process sounds simple and orderly, but we have learned the hard way that design discussions can spiral out of control and become counter-productive. So in order to stay firmly rooted in the reality of shipping software, we don't require consensus to make decisions. However, even we have been amazed at the dozens and dozens of decisions we are able to make every week with little to no controversy.

The ability to 'let go' and 'agree to disagree' is reinforced by our iterative approach to design. Every design decision we make is subject to re-evaluation in the face of feedback and new information. In the meantime, getting something out there for people to start using is considered the critical first step to good design.

Questions we need to answer and the challenges ahead...

  1. What do we mean by open design? (This post is a first step in coming to a shared understanding of what open design means for OSAF.)
  2. What kind of a design community do we want to have? Are we optimizing for the biggest design community possible? Probably not. Articulating what this community looks like in quantitative and qualitative terms is a crucial first step towards building that community.
  3. What is the design equivalent of a code committer? Perhaps a design discussion driver? How many design committers can a single project have without disintegrating into design-by-committee? Does there always need to be a single person overseeing the coherence of the design as a whole? To what extent, if at all, can design be compartmentalized? How do you become a design committer?
  4. What are the different levels of engagement for design contributors? User feedback. Design suggestions. Documentation. Testing. UI review. Coming up with use cases and scenarios. User workflow analysis. Mockups and Specs. Driving Design Discussions.
  5. How do we make it easy for people to engage in our design process?
  6. How do we loop developers into our design process? Code contributors need to buy into our design process.

Concrete next steps...

Establish a solid foundation for the design work we've done thus far:

  • Clear end-user information model
  • Clear design approach
  • Clear target users and target scenarios
  • Visual syntax, interaction heuristics

Build a ramp to engage contributors in design

  • Get users! to use Chandler, so that they can...
  • Provide feedback;
  • Respond to surveys;
  • Log bugs; and
  • Fix bugs!

Build community around participating in use case brainstorming. Make it easy for developers to take on relatively spec’d out designs and engage in design discussions as they work through the details of implementation.

Create room for experimentation. Provide a Design Sandbox where developers can work with relative freedom to experiment with new ideas without having to run every aspect of the design through the design process. In other words, provide a way for coders to start with the features and then iterate by working their way backwards towards real-world use cases and scenarios.

-- MimiYin - 01 Feb 2007

Edit | WYSIWYG | Attach | Printable | Raw View | Backlinks: Web, All Webs | History: r8 < r7 < r6 < r5 < r4 | More topic actions
Open Source Applications Foundation
Except where otherwise noted, this site and its content are licensed by OSAF under an Creative Commons License, Attribution Only 3.0.
See list of page contributors for attributions.