publications

GIMP redux, full GEGL ahead

22 January 2012, 17:36

Recently I have been receiving requests to clarify the GIMP UI strategy surrounding GEGL, so I thought I’d write a catch‑up blogpost about my 2010 LGM lecture. There I tackled this GIMP UI challenge: a first outline for a UI for a fully GEGLed GIMP. The thinking about this UI, and the discussions with Øyvind Kolås (the GEGL‐meister himself), have been going on for years. Its introduction will be the most profound change to GIMP in the foreseeable future.

two minutes of fame

I started off my lecture with introducing GEGL, the graph‐based image processing engine that is slowly, but surely, being integrated into GIMP. I could spend quite some time taking us through the complete GEGL graph (linked) below:

…and this is only a small one. Instead, here is the essence of it in four easy steps:

1. there are input boxes that load an existing image, or render some vector shapes or text;
2. there are chains of operation boxes that perform things like blur or change opacity of whatever gets fed into them;
3. there are composer boxes that take two inputs and put one over the other, combining them in a certain way (think layer modes: normal, dodge, multiply);
4. there are output boxes that either display the grand result on a screen, or export it to graphics file formats like png or jpeg.

so…great

Thus GEGL processes graphics by hooking up the boxes, inputs–to–output. Why does that matter? Well, because it is non‐destructive: the images in the input boxes are never modified.

If the structure of above graph is written to a file—apart from the input images, all other boxes are just snippets of XML—and a year later it is re‑opened in GIMP, then each of the operations and their parameters; each of the vector shapes or text can be freely changed. Even the input images could be swapped out for different ones. The result is a changed image composition, without any loss of quality.

It is exactly this promise of non‐destructive editing that played a big part in me joining the GIMP project years ago. I could see how that could lead to the end of some of the major workflow bottlenecks in today’s graphics software.

UI modelling

The integration of GEGL in GIMP is a disruption; it changes the rules of what one can do and how one can work. This is not a bad thing, it is a refreshing change. An interesting question is: how does the user interaction of GIMP have to change, in order to harnesses all this new power? In general there is a big urge, especially with developers, to just display the graph on the screen:

I call this the boxes and hoses model. If it looks familiar to you, it is because it has been around for decades: it is called visual programming. Which again explains why developers tend to choose this type of solution. One day a direct representation of the graph will appear in GIMP, as something extra. This is because the product vision defines GIMP as (also) being ‘a platform for programming cutting-edge image processing algorithms, by scientists and artists.‘

activity centre

To find out what the main UI in GIMP should be, we take from the product vision what the main activities are that GIMP is made for: ‘high-end photo manipulation’; ‘creating original art from images’; ‘producing icons, graphical elements of web pages and art for user interface elements.’

As a next step, it pays off to look at the nature of these activities. Users start with images or basic shapes (vectors) and apply image manipulation operations, one after another, to achieve the desired result. Users organise their work in layers and GIMP composites the result. Schematically that looks like this:

In short: a work (a composition) consists of layers, each with its sequence (in time) of operations. Now we got the start of a model for the UI. That list of layers we know already, as the layers dialog. The sequence of operations for the layer, we can call that the operations dialog:

disclaimer

Keep in mind that the user interaction shown in the image above, and all the ones below, is not a true mockup. It is more a diagram—with in part grotesque proportions—to show the principles of how the UI works.

We can see that the four GEGL elements are covered: The image material to start with: loaded pixels or rendered vectors; the layers that control the bulk of the compositing; the chains of operations; the output to the screen in the big image window.

Yes, the image window is the place for judging one’s work and for doing the actual work, hands‑on. The image window deserves a couple of times more screen space than any GEGL graph manipulation. Reversing this relationship (as shown earlier) is completely disregarding the nature of the activity.

back to the future

Adding an operation works as before: use a toolbox tool or invoke a menubar item. Here Gaussian blur is invoked and it appears at the top of the operations list:

Now it gets more interesting: revisiting the past. No matter if it was done five seconds or five months ago, one can recall any previous operation applied to this layer and readjust it. Here the curves for this layer are revisited. While adjusting, the output updates through the complete chain of operations, including colorise and blur:

A lot of layers dialog behaviour can be reused for the operations dialog. ‘Eye’ toggles can be used to show/hide the operations:

Drag and drop can be used to rearrange the order of operations. The result of that may turn out to be more subtle than you think. Because GEGL processes everything in 32bit floating point, it will be much harder to get the rounding and clipping artefacts you get with 8‐ or 16‑bit integer processing. Only when an operation does not commutate by nature (i.e. the order matters), then it gets interesting to experiment with the order of operations.

Of course copy and paste of operations, between layers of the same or different files works. Dragging and dropping of operations on another layer copies or moves them, a modifier key (shift or ctrl) will sort that out. And before I forget: selecting some operations in the operations dialog and invoking ‘Make macro’ would be a good, natural way to get that started.

the holy trinity

Before we continue with more GEGL user interaction, here is a pop quiz. I took this b+w photo of the brilliantly named Kloten airport and coloured part of it red:

My question is: how did I apply that red?

1. directly with a tool from the toolbox;
2. making a selection, then invoking a menu item;
3. using a layer with a layer mask.

Well, I have forgotten how I did it (it’s been a while) and you will never guess. My point is that it does not matter. All three methods listed above are a combination of an operation (apply red) and a greyscale mask, controlling where it is applied and how much. All three methods are fully equivalent, it does not matter to the software—or the viewer.

It does matter to users. Given the context, composition structure, the graphics material and the end‑goal, each user knows exactly which of the three methods she/he prefers. There is a million different use cases and only the individual user on the spot can take the right decision. It follows that each of the three methods is equally important and needs to be equally available to users. This I call the holy trinity of image manipulation.

pagan practices

However, at the moment, you quite often see the following: ‘if you want this feature, you’ll have to use it on its own, extra layer.’ This is layer abuse. I get misquoted on this so let me clarify: users never abuse layers, developers do. Here are some examples of layer abuse:

• the only way to do a non‐destructive operation is via an adjustment layer
• only one vector shape per vector layer;
• only one block of text on a text layer;
• the output of a filter plugin is always put on a new layer;
• the result of using a toolbox tool is always put on a new layer.

The problem is with ‘only,’ ‘always’ and ever more layers, whether users want them or not.

reformation

The abuse listed above is straightforward to fix. Quite a bit of it has to do with enabling users to redo or revisit the image manipulation. That is solved by the operations dialog.

Furthermore, there can be as many vector shapes and text blocks on a layer as one likes. Just show them—and stack ’em—as sub‑layer elements in the layers dialog. And when then one of these sub‑layer elements is allowed to be actual pixels, then it is clear that the whole notion of special vector/text layer can disappear:

Layer abuse has to stop. Developers should never force users to use another layer. Only users decide how many layers they want to use, purely as their own personal way to organise their work.

more blessings

So apart from that no image manipulation is exclusive to a layer, what are further implications of the dogma of the holy trinity?

paint with anything

Yep, with anything, also all the plugins that appear in the Filters menu. Also the obscure ones that you and I, and anyone on the GIMP team have never heard of. Just dip your brush in it and smear it on the canvas. Thinking of combining it with paint dynamics just blows my mind. It just takes one tool (Filter brush?) to enable this.

adjustment layers with anything

You see, I have nothing against accomplishing things with layers. It is perfect when you got a collage made out of dozens of layers and then want to apply some treatments to the whole thing. It just takes one adjustment layer with any number of operations to get that done. Again, with anything.

behind the mask

As mentioned, all three image manipulation methods consist of an operation and greyscale mask, controlling where it is applied and how much. The interaction for these masks can be analogous to that of layer masks:

This means that both the parameters of the operation and also the ‘where and how much’ can be revisited and adjusted, a second later or a year later.

in triplicate

By now you may be thinking ‘wow, every image manipulation must be possible in three different ways, does that mean that one day GIMP will contain three times more stuff than today?’ Well no. I have already mentioned two measures—Filter brush, adjustment layers with anything—that will make large strides towards fulfilling the holy trinity. And even today the trinity is partially in place at GIMP.

Take for instance composition, putting pixels on top of pixels. Sounds like the exclusive domain of the layer stack, no? Well, even today you can use toolbox tools to do the same thing: the Clone tool and its 90%‑identical cousin, the Heal tool. And the menu item to compose is called Paste. Yes, the current layers dialog interaction while pasting should go. The GIMP team is agreed on this for years. Instead paste is an operation on the current layer:

weird science

There is a fifth element to GEGL graphs, that I have been keeping under wraps in this blogpost until now: cloning. This is taking an in‑between result in the GEGL graph, anything from a simple vector to a huge sub‐tree, and ‘teleporting’ one or more clones of this result to another position in the graph:

The original and the clones can have the same or a different position on the canvas; undergo the same or completely different operations; be composited in the same or a completely different manner. The magic is of course in that when the original (the input to cloning) is updated, every clone immediately updates, with all further operations and compositing applied on top. This is going to be liberating, because of the amount of work this saves.

Let me give an example. Let’s take the scan of a snowflake and fix it up with a number of graphics operations. We clone the result 99 times and spread these over five layers to compose our snowscape. We make ever flake a different size and a different colour. Already laborious, no? And now, we decide we are not satisfied with how refined the snowflakes are. Solution: fix the original with more/less/different/updated operations, all 100 flakes will immediately show the improved refinement—in their different sizes and colours.

send in the clones

My plan for the user interaction of cloning is to make it a variant of pasting: Paste as clone. This includes doing further operations on the pasted (as clone) material, before closing off the paste (you can always revisit the paste operation and modify all of it). And as Ville Pätsi pointed out a while ago, there needs to be a way from each clone to access the original. A button or link on the Paste clone operation in the operations dialog might do the trick.

There is more tricky stuff coming up with this cloning, like cloning layers or layer groups. And to go fully meta on this: I am asking myself why chains of operations cannot be cloned and applied to different input material. But Øyvind is not up for that.

two more things

To wrap all this up I have two benchmarks that have developed out of the GEGL discussions. They are alternate manifestations of the holy trinity and they are serious tests of the direction the GIMP user interaction is taking.

First, in a future version of GIMP, if users really envision their work as one single layer—not that unusual with photographs—then they will simply be able to close the layers dialog. Their work will not be impeded in any way by this, everything is still available to them (twice, as toolbox tool and menu) with no limits in the sophistication with which they can express themselves:

Second, in a future version of GIMP, if users are really not interested in switching operations on/off; rearranging operations or revisiting the past, then they will simply be able to close the operations dialog. Their work will not be impeded in any way by this, everything is still available to them with no limits in the sophistication with which they can express themselves:

You could even combine the two benchmarks and GIMP would still work, without a hitch.

aftermath

Directly after the lecture at the LGM, Øyvind Kolås pointed out one flaw, one thing I had not thought of: those greyscale masks, that play such a central role, are not pixmaps. They are GEGL sub‐graphs themselves. Which creates the possibility of near‐endless drill‐down: operations with greyscale masks, which contain operations with greyscale masks, which contain… et cetera.

Is that a fatal flaw? Is that going to blow up my plan to reduce the complexity of the GEGL graph to a linear list of operations? Do we just have to confront users with the whole graph? Let not kid ourselves about the complexity of that graph. In practice, I’d say that a simple GIMP file starts at ten times the number of boxes in the example at the top, quickly moving to a hundred times and then beyond.

not my problem

To developers—especially the ones that are working up to their elbows in GEGL graphs—this problem looks like one of navigating the graph and making edits. But to me it looks like another problem needs to be solved: how can users navigate their working context? Already today GIMP users are doing this navigation. Which layer, which layer mask are they editing, or is it the quick mask?

It will have to wait for another day, but when this context navigation is going to be designed, the one thing not to lose sight of is the image window. Everything evolves around it, including this navigation, and feedback of the working context has to be inside it. I certainly think we can do a better job than currently happens with the layer masks.

And when this context navigation has been designed, taking into account the holy trinity and the two benchmarks, then the operations dialog can show the simple list of operations performed in that context, in their natural order.

Labels: GIMP, GIMP redux, lecture, practical, product vision

—ps

30 comments, link‐ins, forward this posting:

mobx, the big bang

30 November 2011, 19:47

And now for part two of my MobX lecture. In part one we saw how old mobile was becoming bloated, more complex and slower. And then…

boom!

January 9th, 2007 the iPhone was introduced. It was the big bang of mobile. Looking at that first version hard‐ and soft‐ware today, it looks quite humble:

During a year or so after the introduction I caught up with my contacts at mobile manufacturers and networks. I found out that the response had been universal across the industry: ‘oh shit.’ This was followed by an investigation, ordered by the highest company level, into ‘why did we not invent this?’

culture club

That is a question of culture. All these companies have many very bright researchers. The question is what is being done with their work. That is a management problem. As is excellence in execution. Good enough is simply not good enough, today.

We also know that Apple had to keep the mobile networks out of the loop to make sure that the iPhone would not be any old mobile. At the end they found AT&T prepared to sign up, sight‐unseen, for the iPhone. After the introduction the genie was out of the bottle and no mobile network could put it back.

touch rules

Mid‐2007 I was for a couple of weeks in the US to work on the interaction fundamentals of an ‘iPhone killer.’ It was my first chance to get my hands on an iPhone, introduction in Europe was in November of that year. It took only a few days to figure out the rules of touch (apart from the obvious changes):

• The user interaction is more within a given screen, instead of being defined by transitions, as it was for old mobile.
• Use the whole screen. Whatever you do, make it as big and gorgeous as possible.
• Pixels are no longer the unit of measurement; it is the finger grid in centimetres.
• Touch interaction is addictive. Especially sliding, swiping or flicking things is the stuff that joy of use is made of: satisfaction guaranteed.
• No Options menus anymore. They are virtually gone.
• Back‐stepping (pressing the Back softkey) was such a natural part of old mobile that it was not even shown on the chart. With touch it is mostly avoided.
• This is not a phone. It is a handheld computer that happens to make calls.

In the two years that followed, I led the interaction design work on more than one ‘iPhone killer,’ for corporations that were simply not getting it. They were still insisting that their devices should have green and red hardware keys, as if they were making a telephone. Usability folks at these corporations were not exactly helping the innovation, playing it safe by insisting that everything should be operated by means of simple tapping. What was that about swiping?

like a robot

Now that we know the rules of touch, can someone tell me what is going on with android? April last year I was at a conference—nothing to do with mobile—and Google gave away a nexus one to every participant. It took only a few days to figure out that something was really wrong. They’re back:

The return of the Options menu and Back hardware keys is not just a case of old‑think. It is a serious problem. This is because the bright and shiny display is where the interaction is, and these two buttons are not part of it. And that is how users are perceiving it, they always pause when their next step can only be performed with either key. They get torn out of their interaction flow, press the key, then re‑enter the flow.

It is the law of the platform and android is stuck with it. But those of us who design user interaction for android apps can help. Why not put all the interaction on the screen? Refuse to depend on these two keys. And when the guidelines of android still prescribe to make use of the Options menu and Back keys, then treat them as a backup solution.

This is analogous to the situation on the desktop, where for instance in iPhoto most of the interaction can be performed in the main window, and the menubar is there as a slow, but safe, interaction element prescribed by the guidelines.

glass half full

What about hybrid touch, aka touch and type? Is it half‐baked? Looking at the image below it is obvious that the bottom half of this devicee cannot be used to watch a film, read a webpage or play a game. But, you can type like it’s 1999.

When I first had a prototype for this device in my hands, I thought: ‘wow, finally something is not trying to be an iPhone.’ This one is doing its own thang and deserves some love. The UI however is a direct adaptation of old mobile, adjusted for the obvious changes: direct touch input; no soft‑ and scroll‐keys. This must have really helped in the feasibility department, but it does the device no justice.

but there is hope

Early last year I led the interaction design for a software project for these devices. Without going into the project, I would like to show you that there are opportunities to push the envelope. Ignoring what is on the screen, let’s concentrate on the softkey line:

That softkey line is a touch area, of course. And because it is on the bright and shiny display, it is usable, unlike the android keys. Being touch, you can do whatever you want, which I did:

I put three icons (placeholders shown here), giving users direct‐access navigation between three sections of the application, as seen on touch devices. Together with the remaining right softkey, it is a truly hybrid solution. I kept the boss of this UI platform in the loop and she was very encouraging. She was eager to see innovation on the platform.

So do not despair when you have to work as an interaction designer on a hybrid touch project. It is a grey zone which means lots of opportunity to define it yourself. Go on and break the rules, and innovate.

the future of old

Looking at the graphs below, from the incredibly useful asymco blog, we see both for the US (left) and globally (right) smartphones—today most of them touch devices—are definitely on the rise towards dominating the market. Note that in our context both symbian and RIM are good old mobile—and were never that smart—but they follow the trend and are in decline.

source: asymco.com

Before you now predict the death of old mobile, let me point out that something as simple as component costs will keep proper touch out of the hands of most of the world population. Serving the worldwide middle classes: yes; everyone: as soon as the total parts cost is $10 or less.

So old mobile is here to stay, for a while. Let’s not forget outside the western world there is a lot of service innovation going on that puts the western world to shame. All of that is on old mobile. And no, working on old mobile does not need to be nagging legacy support.

dual strategy

I have blogged before about working on Nokia’s first dual‑SIM phones. I can talk about this project for hours, but what you need to know is how the fundamental equation of mobile phone was changed from one user, one SIM to one user, two SIMs. This triggers a massive engineering effort, every software component in the phone is affected. And if you don’t watch out, then for the UI you are up for redesigning hundreds of screen layouts, implementing these and re‑specifying all software that use them.

Because of this massive effort it was not that sure at the beginning of the project that we were going to make it. I was the interaction lead for the whole phone. I created the solution strategy and accompanying design patterns. Then I worked with Nokia designers and mentored them while implementing these patterns. And we made it:

And that was in part thanks to the solution I designed. My top priority had been good user interaction, but a happy side‐effect was a great reduction in UI design effort and overall engineering work. In May this year the first Nokia dual‑SIM model appeared on the market; today there are five models.

When you look at the phone above you see it is good old mobile. The solution could have been made in 1997: it is all just lists and Options menus. And the fist device we designed it for is low spec mobile, with just about the smallest screen size Nokia ships.

the future is bright

The dual‑SIM project turned out to be one of the most satisfying projects I ever worked on. And it was not just because of the challenge, the mind‐boggling complexities; not for the travel involved or the rewarding cooperation with my colleagues. It was so satisfying because all the time we worked on it, we knew how much it mattered.

How much it mattered for Nokia’s business was proven a couple of weeks ago when its third quarter results were released. In between the gloom and doom there was some light. They had shipped 18 million dual‑SIM devices that quarter. Through this they had gained market share in India. Even though with dual‑SIM we are talking about cheap devices, that is around a billion dollar in turnover.

In the financial statement and all marketing material, Nokia proudly communicates that their dual‑SIM solution ‘enables users to personalize up to five SIM cards.’ Cool, I designed that. The personalisation: instead of calling the SIMs just A and B, let users name the SIMs for what they mean to them, like ‘cheap calls’ or ‘free SMS.’ And that number five, I set that as the minimum to remember. It is quite normal for users in these markets to juggle three SIMs every day, and to change one or two of them, every month. 3 + 2 = 5.

Just the other week Nokia CEO Stephen Elop spoke at a press conference of the ‘halo effect’ that dual‑SIM has in the market. All sorts of Nokia devices are selling better because dual‑SIM is now shipping.

users + nerds

Also on the user side we knew how much the dual‑SIM project mattered. There was a pent‑up demand for this and now it is safe to say that out of those 18 million people that bought these devices, 100% did it for the dual‑SIM. And 100% of them are using it every day. Compared to the one percent I mentioned in part one, that matters.

Reviewing all this, I am thinking: maybe this touch UI vs. old mobile is all relative. Maybe it is just designer nerd‐talk? It reminds me so much of discussions developers have about the coolness and merits of doing a project in C, C++, Java or something new, like lua. Just maybe, it does not matter that much.

So for us designers, it might be more rewarding to be excited about how relevant the project before you is for your users and the business of your client. And at the end, you will know you have worked hard enough only if you are excited about how elegant you made the interaction, and about how much innovation you put in.

postscript

That’s it for my lecture. I could only share 5% of what I had lined up in my mind, but there was only half an hour on stage for each speaker and time goes fast. I am looking forward to the next opportunity to share some more.

Labels: fundamental, lecture, mentoring, mobile, practical

—ps

1 comments, link‐ins, forward this posting:

mobx, mobile before January 9th, 2007

28 November 2011, 22:22

Hot on the heels of the WUD there was the MobX conference. Very cool to be invited for this one, the speaker lineup was promising and delivered. The feedback of the conference audience was enthusiastic, both on the day itself and via all the familiar networks. As usual, here is the write‑up of my lecture.

mobile before and after January 9th, 2007

I started in mobile user interaction design as a contractor at AEG, in early 1997. At that time, mobile looked like this:

source: handy-sammler.de, thanks for the memory!

The pixels on this mobile’s screen were not square, they were rectangular; more high than wide. This was to save on screen cost and on memory needed to drive it. What I learned quickly is that a mobile is a complete computer, with an operating system; network communication stack; UI frameworks and applications.

More fun trivia: I did not own or use a mobile in those days. Only two years later I got my first mobile. I had seen mobiles before, more of them in the UK in ’95, less in Holland in ’96. But I had never made a call with one.

spec work

Work on mobiles was in those days driven by the GSM specification, which regulates how mobile networks work and type‐approval of mobiles. These specs are written by a committee of manufacturers, but I learned that they did this in order to please the network operators, to create new revenue streams for them.

The SIM toolkit, which I worked on in ’97, is an example of that. It lets networks add menus and UI to mobiles, for proprietary services. Speaking of services, in those days SMS was used by virtually no one, it was simply too expensive.

touch and go

Looking back it is quite amazing that we were working in ’98 on a touch UI model, for production. OK, it looked more like the model above than an iPhone, but still it had handwriting text input. Alas, that work stopped in the mid‑1998 when AEG got out of the mobile business. Nokia had invented segmentation—different mobiles for different people—and were solidly beating companies like AEG who were making only one model at any given time. Segmentation seemed to be the way of the future…

a cup of joe

We now fast‐forward to 2001, when a long stretch of mobile work started for me. Nokia was about to start working on their first java‐equipped phone, the 6310i:

This model still had a 1‑bit, black and white screen; two softkeys and 2‑way scroll keys. I still see the 6310i in use today, nine years after its introduction. They were built to last.

Mobile models were at that time differentiated by ‘extras’ which had taken over from the GSM spec as driver of development. I was asked by Nokia to design the first java mobile applications (MIDlets) to ship with the phone, together with a Nokia colleague. From this phase of ‘old mobile’ I would like to discuss three things I learned that are still valid in today’s touch‐dominated world.

1. work on a chart

This is the UI chart of the first MIDlet I designed, the converter. It converts units, like pounds to kilos and gallons to litres:

The point is that this is the complete user interaction. It is accompanied by an error chart—derived from the UI chart, it shows what happens when things go wrong, e.g. memory full—and a short document, less than ten pages, that specifies details. This is all you need to develop and ship the MIDlet.

Case in point. A couple of years later I worked on version two and three of Nokia’s java email client. Version one reached me in the form of a 189‑page document. Text‐only: a section for each screen, with gotos at the end for the transitions. It had taken nine months to write and was unusable. I spent a week decoding the document and made this chart:

Now Nokia knew what they had been developing. The chart was my first deliverable and it was a revelation to them. Worth every penny, they thought. In a much shorter timeframe, nine weeks, I did a redesign for version two, which had a lot of new stuff included. The result had to be put on two charts, though.

incomplete story

Every once in a while, I see storyboards being used to communicate an interaction design. You know, a row of screens showing what happens in sequence:

This may work—even well—to present work to people who are not in on the project. But it is not a working tool for interaction designers. It is not fit for communicating the design to development. That is because it cannot show a complete and coherent picture of the interaction.

seeing is believing

What I learned after a while is to read the chart. To see the quality of the user interaction from the patterns. This works because in old mobile, a lot of the interaction is not determined by what is on the screens, but by the flow between the screens. Here is the basic flow of old mobile, there is not much more to it than this:

1. Without even looking at what is on the screens, you just know that this is the Options menu.
2. Under it are the task flows, each completing a task.
3. The return point for the flows. After a while I sensed that this is a point in the UI where users feel calmer, more secure.
4. The flows themselves are more uncomfortable for users, they do not like to hang out there.
5. Longer flows spell trouble; one screen more and I would be alarmed and investigate. Note that this is about feeling (see 4) and not about number of clicks (a most useless metric).

If a chart has arrows flying all over the place, then either you have not drawn it very well (try again) or the user interaction needs a redesign. This is why I like to have the complete and coherent picture. To see how everything is connected and relates to everything else. That’s why I say: work on a chart.

2. there is no such thing as cross‑platform

I learned this one the hard way. So there we were at Nokia. There was a java team that had to make a toolkit that would run any MIDlet in the world. And there was me asked to design real Nokia applications using this toolkit. This was a constant struggle.

I do not blame the java team. They had to adhere to a common‐denominator specification and do the best they could. But the loss of control on my side meant too many cases of close but no cigar. The make‑or‐break details of user interaction could not be put in order.

difference engine

After a while I was also designing for multiple devices. All Nokia devices, but the differences started to bite. It is easy to see the big impact of changing the number of buttons on a device. From two to three softkeys; introducing 4‑way scroll.

Remarkably more buttons was not always better: I had optimised the converter so much for two softkeys, that when the third came along, the problem was what action to assign to it. At the end I put the conversion selection, but it was clear that the converter had not been aching for three softkeys to come along.

size matters

More subtle were the differences in screen sizes of the devices. This meant changes in how much text or how many items one could fit on a screen. This situation is quite elastic, but sometimes there would be a catastrophe because something would not fit anymore or an opportunity because an extra thing would fit.

This would trigger a redesign, and I do not mean just of some screens. Whole flows had to be changed because of the catastrophe/opportunity. I think today with touch UI this screen size factor is even bigger than with old mobile. Pixels sizes do not count anymore, it is the finger grid, measured in centimetres. That means less elasticity.

So when somebody says ‘we are going to do this cross‐platform from one codebase’ or someone else says ‘I got a cross‐platform UI toolkit,’ then I say: no you don’t. You have no idea what user interaction is really about.

3. be obsessed with elegance

So there I was, designing these small MIDlets that really only did one thing. And I started to be obsessed with how elegant the interaction could get. An example is the Nokia translator, which translates a couple of thousand words between five languages.

This looked easy, but there was one thing that bugged me: should users constantly be telling the translator what they are translating (German–English or English–German?). Language is a two‑way thing, so this directionality had to go. I was obsessed with this for more than a week, day and night. But then I had it.

Language is the answer. Just look up the input word in all five dictionaries. Spelling will sort out in which language it was. We ran some tests that confirmed the plan. The result was a leap in elegance. And a small, very satisfying UI chart.

complex issues

I used to have a complex, designing these small MIDlets that really only did one thing. Other people were working on bigger things like call, phonebook or SMS that were also used by a lot more users. I have always estimated that one percent of users used these small MIDlets that I designed. With an installed base of half a billion that still gives five million users, which is not bad.

These days, small programs that really only do one thing, and are used by only a fraction of users, are called apps. They are perceived as beautiful and people have no hang‑ups about working on them. And elegance rules this domain.

grinding halt

With hindsight it is easy to say that being small is beautiful. This is because I can now see that around 2005 something was clearly going wrong:

Meet sensor, today you would call it a social network, in this case a short‐range one: it worked via bluetooth only. It is so large it needed four UI charts; you can feel my pain. I redesigned this from the original symbian version. It was quite a bit of work, because there is no such thing as cross‑platform.

This is really exemplary of what was happening at the time. Mobile was getting bloated and more complex. Users were noticing it. Everything just felt slower. Straightforward things were no longer straightforward.

That is, until January 9th, 2007. Read about it in part two.

Labels: fundamental, lecture, mobile, practical

—ps

1 comments, link‐ins, forward this posting:

the wud, on social networks

15 November 2011, 18:10

I have just just wrapped up another world usability day (WUD). The theme was Social Networks and we had a full house—workshops and lecture hall—all day. As mentioned before, I had hands‑on interaction design experience with a sprouting social network to share in my lecture. A video registration (in German) of the full lecture can be seen online and, as usual, here is also my write‑up.

structured visualisation or simply being social,
the design work at unusuals.net

unusuals is a startup in Berlin that runs the online network for advertising film professionals. At the beginning of this year they were working on their relaunch and approached m+mi works to help revamp their user interaction. We agreed on a ten man‑day consulting project, which was then implemented over a two month period.

With a modest project like this one cannot expect interaction architects to make and document detailed designs (wireframes) of a whole social network website. There is simply not enough time; practical consulting is the most effective way to shape the project. Fortunately, the unusuals were up to the task of wireframing themselves.

just getting started

The work before the work that I do at the start of a project is structuring. At unusuals I implemented the following sequence of methods and phases:

product vision

Define of what is it that we are creating; what is the end‑goal.

functionality

Create a complete overview of ‘what’s inside the box.’

user scenarios

Further work out the vision with scenarios showing typical and essential use. These are then used for…

expert evaluation

Evaluate any existing stuff: ideas; plans; designs; software; websites; competing offerings.

brainstorming

Time for creativity and generate new ideas. Starting with real life; take it through a couple of loops of fantasy; then back to earth.

design sessions

Sit down and solve the design problem at hand. From fundamental stuff like browser sizing and flexible layouts; to mid and detailed design of components like a film gallery.

This may look like a long‐winded process, only getting results at the end (the design sessions). Let’s see whether this was really the case.

panavision

We started—as is my custom—with the product vision phase: defining what the product is, its identity, what the target user groups are and where the value is. This took a couple of hours with unusuals, especially because they had to explain to me the advertising film world.

How many people does it take to make a tv commercial? Easily more than 50. All these people work in a pyramid, in a strict hierarchical system. It is also a world full of freelancers and the name of the game is getting that next job; then recruit the people who work directly below you in the pyramid, within one or two days.

keep on rolling

Two weeks before the WUD, the unusuals kindly agreed to help me prepare by spending two hours recounting the project. During that, they admitted that they did roll their eyes (a bit) when they realised they had to explain the film world to me, an outsider. They themselves have 10–20 years of experience in the business as either producer or director.

But then they also quickly realised, while explaining, that this was helping them, formulating this simplified version to an external. Also my questions helped, always pointing in the same direction: how does this film world tick. Because how it ticks is how the unusuals network should work. And it is the interaction design that defines how it works.

this header has been intentionally left blank

Now I would like to show the unusuals’ product vision to you, but it is simply too strategic. User interaction, being so directly connected to it, is that strategic. But I can share some of the keywords of the vision. unusuals is:

international

There are a hundred thousand, or two, people in this industry, worldwide.

transparent

The whole point of the network.

exclusive

For insiders only.

a ‘kitchen party’

Thinking about identity, the unusuals wanted the opposite of industry (awards) events, with its cynical networking. They wanted to have that vibe of a private party at someone’s home, full of film industry mates of course. And the best place to be at a party is… the kitchen.

just can’t get enough

Where it came to functionality, there was already a wall full of post‑its; the proverbial wish list of everything other social networks do, plus some more on top. But luckily there were some moderating factors. There was a fixed (re‑)launch date of mid‑June (real artist ship) and there was also a planning, so they knew they had to do a feature freeze way before June. Apart from that I also encouraged them to focus on the essential, because squeezing it all in would be in direct conflict with achieving great user experience.

And here I must make the unusuals a big compliment. Unlike many other companies, they were able to prioritise functionality, to decide what really mattered. And ensure that that would be designed and implemented to a high standard. They were able to draw on their experience as film makers and translate that to the software maker world.

But also they had a new basis for their decisions upon. They had their product vision. So instead of individuals fighting for their feature, they were a team checking if features were essential for reaching their common goal. The unusuals completed this method as ‘homework.’

internet‐free zone

More homework for the unusuals was writing their user scenarios. I say: six is the magic number for user scenarios. I also had an extra assignment for the unusuals: keep the technology out. No mention of social stuff like poking, liking, following; do not even mention the internet. The focus should be on users and what they are trying to achieve.

Being natural storytellers, the unusuals gave the main character of each scenario a name. Here is a swift impression of a scenario: Pablo searches for information on film‐making companies; Pablo asks his friends for some tips; Pablo looks at films to get inspired; Pablo checks the details of a film company; et cetera. Usually around ten steps like that per scenario. Only human tasks and needs.

fun fun

The unusuals said they had fun boiling an initial fifteen scenarios down to the magic number. Seeing the commonalities between two of them and merging them into one. And they learned about what mattered to users while doing this.

This method overview shows that designing good user interaction did not have to wait until the design session. It started right away, from the first step, discussing the product vision. Let’s now take a look at how those four vision keywords drove the interaction design.

up and away

Starting with international, yes it is a worldwide industry. But it is also very local. That professional is needed here and tomorrow. In real life everybody maintains a local network of contacts for that. And there is a third factor in this paradox: this industry has a penchant for exotic locations. And with that a couple of questions rise:

• What network do you tap while filming on location, far from home?
• When a professional wants to base himself in a new town, how to get started there?
• Even simpler: what do you do when all the usual suspects who can fill a position are busy next week?

The unusuals network is of course the answer. We combined the transparency that it needs to deliver with the local‐but‐international paradox to arrive at a design where one page handles many tasks:

• organic exploring, starting from this user’s own network;
• organic exploring, starting from anybody else’s network;
• organic exploring of the filmmaking market of a city, country or the world;
• recruit people, starting from this user’s own network;
• recruit people, starting from anybody else’s network;
• recruit people within a filmmaking market of a city or country.

Conventionally this would lead to up to six different web pages, each designed differently and fragmenting the experience. Shown below is how we designed it: in a natural, unified—if at first sight maybe even unspectacular—way. Starting with exploring:

Adding filtering for a certain role:

Or take as starting point the network of people one trusts, or a certain market:

What we see is user interaction shaped by working methodically; combining the product vision, user scenarios and the understanding of how users tick.

splendid isolation

unusuals being an exclusive network is a great thing. It really helps to focus while designing, creating a place where this user group will feel right at home. Which can only help to build the network. But it will also help the network to survive.

What is it that destroys social networks? It is that cool innovators—who are the gravitational force that builds a network—do not want to be seen in the company of people who they consider ‘clowns, jerks and definitely uncool.’ So they leave networks when they feel overrun by these types, adopting a new, hip hangout for themselves. As usual, this is followed by an exponentially rising tide of invites that builds this next network.

lean and mean

My conclusion is that all general networks must die. By which I mean that these networks have the same life cycle as boy bands: every couple of years there is another bunch that dominate the charts. It is just a fashion/trend cycle, driven by these cool innovators.

I believe specialised, exclusive networks have much greater staying power. By keeping out people who are definitely not insiders, reasons to leave can be kept below the threshold of the cool. This why I am happy that the unusuals followed through with keeping it exclusive:

To join unusuals you have to be invited. And to be invited you have prove your credentials. Modelled on real‐life strategies of getting past the doorman of an exclusive club, there is two ways to do it. Either ‘do you know who I am?’ or ‘people are waiting for me inside.’

you can’t touch this

What became of the kitchen party? There is no part of the unusuals site that can be identified as its tangible representation. But as the unusuals pointed out in their feedback, it is everywhere. It certainly got mentioned a hell of a lot throughout the project. That helped with finding the right form while designing and also helped with important decisions.

Having you own offbeat concept like the kitchen party really works like a conspiracy within the design and development team. It does keep everyone pushing in the right direction.

paper + clip

With the kitchen party comes the curious host, who keeps the party going by discreetly asking a question or two to participants and using that to establish new connections and interactions. Again, this is not directly tangible. Do not expect Clippy, the curious host.

Instead, it resulted in a policy: no form‐filling, beyond the bare minimum, like on the request for an invite. There is no duty for users to reveal information about themselves, only opportunities. ‘See that cool sneaker spot? Yeah, I edited that.’ You can then count on the vision of a exclusive, professional network; that users will take the opportunity to profile themselves.

talk talk

And now for some social interaction. Among all that initial functionality on post‑its there was all sorts of communication. Private messaging, group discussions, comments, following, feeds, tweeds, et cetera. Each of these looks ‘easy’ to add. In reality we underestimate the effort because they are ubiquitous, infrastructure type of functionality. We tend to forget about all the details that need to be designed and implemented for each.

Part of my job is to exactly point out these implications. And to show how much work it would be to integrate these with each other, as well as having to explain to users why all these have to exist within unusuals. For example, private messaging is covered well by good old email. No need to duplicate it. It was time for me to say: ‘Pick one. Which form of communication is the most important for your vision?’

crashing the party

Fast forward to a round of brainstorming, where the vision of transparency was combined with a look at how people communicate at a real‐life kitchen parties: two or more people conversing with each other anyone standing close enough can listen in. And anyone who has something to contribute can join the conversation.

That is how conversations were designed at unusuals. Starting with a comment to a film, or person A starting talking to person B. When someone answers it is a conversation, anyone can join to broaden it. Nice detail that there is no ownership of any given conversation, they are simply attached to all persons and content involved.

the right stuff

The best part of this is that the unusuals own this stuff. No, not like in software patents. I mean they worked it out themselves. The worst you can do when developing user interaction is blind copying: ‘facebook has a wall; so must we. They are quite successful.’ Not really much better is the ‘not invented here’ attitude. ‘facebook has a wall; that will not happen to us. We must invent something unique.’

The correct way to develop user interaction is to work through the methods outlined above, beginning with that you’ve got to have a vision. If then your design is quite similar to something already existing, then that is OK. You arrived at it independently, from your own strength.

During the unusuals development period this year, facebook changed the behaviour of its ‘wall.’ Two weeks ago I asked the unusuals what they thought when that happened. They said that they could not care less. They have their own thing, which is right for them and their users. They have their independence.

red carpet day

Further brainstorming produced premieres, of course based on the film business. It solves the problem of how a crew that worked on a film can be kept up to date of the release of said commercial. unusuals lets members schedule a premiere of a new commercial. Those closely involved with its release will do so, out of pride and/or as part of customer service. Via invitations and organic networking the crew gets to hear about the upcoming premiere.

And here the curious host principle pays off. Any member checking out the premiere has to opportunity to put himself with a simple action on the crew list of the film (yes, there is peer review to fight abuse). This action (soft‐)connects the crew: they have worked together. We see again how important content is for starting social interaction and building the network.

quattroporte

The unusuals had a dream. They asked me if the site could feel really simple. Like only needing four departments to structure the whole site. Problem was that at that time their straightforward classification of their functionality called for nine departments. So I thought about it for a short while and the answer is: yes, you can.

There is no standard recipe for achieving this simplification. Solving it is quite an intuitive design process. Full understanding of the product vision and how users tick goes into it and suddenly, it gels. The understanding that exploring the network, keeping up with social activity and users presenting themselves were the most promising categories to put top‑level. Then the functionality overview is used as a work list to check and place everything:

Top‑left is the Explore section we saw before. The Me section is all about this particular user (profile, settings, media uploaded and associated with) and the Buzz covers anything social and dynamic in time. Finally the Tools section offers room to expand. For instance managing premieres belongs there.

get the most out of it

Phew, already a long haul and it still only covers part of what we were able to achieve during this modest, ten man‑day project. So I asked myself: how come the unusuals got so much out of this? I think it is because:

1. The unusuals were fully open to process change. Introducing proper interaction design means that things cannot stay as they were before. Because unusuals is a startup it was more a case of process shaping, but still they got it without a need for heavy discussion.
2. The unusuals reached out to get a professional on board to work on their user interaction. And they were determined not to go it alone. They kept looking until they found a partner that fitted their needs; then without further ado started our project.

And here we have the acid test. Without these two, really nothing is going to happen. You cannot bolt on some usability or interaction design consulting to the side of you regular development process and expect results. For that they have to become the core of your process. Similarly, doing serious work takes experienced professionals that you trust. When both of these points are not in place, then all that is left is talk, about improving usability and having a great user experience. Read more about process integration—from last year’s WUD.

What also really helped is that at unusuals I was working directly with deciders. People who, without distractions, put their 100% in making the best product they can.

live and learn

A nice aspect of my job is that you always learn something new, it keeps things interesting. What I learned from the unusuals is that especially for startups, structuring is more than half the work—I explored that deeper in a previous blog post. They stressed that they really valued the methodical approach to the project.

The second thing I learned takes that to the next level. The unusuals insisted that the early methods—product vision, functionality, user scenarios—liberated them. They could stop talking about unusuals on other peoples terms, start with a clean sheet and express what and how they wanted to create themselves.

This really bowled me over, I had never looked at it in this way. I use the methods for information transfer and as a solid foundation of the interaction design. That these methods can work emancipating for my clients, well, I keep that in mind from now on.

the ins and outs

The final lesson learned is that a social network like this is a living thing. The growth and dynamic of the network definitely forms another channel of feedback that goes into the evolving interaction design.

Working with the unusuals was a blast. I am proud of what we achieved in such a short time. Normally this would be the point where I say: check it out, unusuals.net. But the chance is high that you are not in the advertising film business. So you won’t get in. But that is actually a good thing.

Labels: lecture, practical, product vision, startups

—ps

0 comments, link‐ins, forward this posting:

12 interaction tips for startups

10 October 2011, 14:03

Over the last year or two, we at m+mi works have seen a remarkable increase in projects with local startup companies. For us this is tangible proof of the alleged trend that Berlin is the location for your next startup. I can see the driving forces behind it though. Having set up shop myself in Berlin, I can only confirm the advantages and benefits of doing so.

Comparing our experience of working with startups with that of many years of working with clients great and small, I have noticed commonalities and some differences, and have digested these below. If you are running a startup that is working on a product that has user interaction, check out these tips.

1. do you really want to?

Do you really want your product—application, mobile app, website or online service—to be compelling and elegant to use? I have to ask, because a lot of folks I talk to say they want this, but in reality very few take the necessary steps to make it happen.

Starting with the right mindset (is the value in features or in user experience?); followed by finding the right people to design your user interaction and putting them in charge; then, when it is crunch‐time, standing by your user experience goals, instead of declaring them ‘nice to have.’ This is just a selection of what it takes.

If you are wavering at this point—or think you can cleverly bypass one of these steps—then you might as well stop reading, with the certainty that your product will never have a great user experience. Maybe you can make it regardless (‘if no competitor bothers, then…’). However, if you know that your success depends on your user experience, then read on.

2. you have a rare opportunity

At startups we regularly find the kind of environment that is conductive to the design of great user interaction. Working directly with the top decision makers; a lack of conservative and reactionary forces; genuine believe in the product; relentless focus on product inovation; a ‘can do’ mentality; knowing that only the best survive.

Add to this a product idea that must be innovative (‘me too’ or ‘feature parity’ equals death) and you can see why it can be exhilarating for us at m+mi works to work with startups.

My message to startups: you owe it to yourself to make use of all this energy. Unlike most other companies, you have the opportunity to produce something truly innovative, with user interaction that sets new standards. Grab it.

3. yet another development company

In some regards a startup is not that special. It is simply a software development company, which is governed by the same ‘laws of nature’ as any other software company. It is not that working at a startup gives a divine shot of inspiration and ability to developers, visual designers and product people for creating user interaction, is it?

The laws of nature dictate that if you want your product’s user interaction to be a hit and drive the growth of your company—instead of impeding it—then you need to involve specialists. No, the creative guy with photoshop is not it.

Similarly, going around friends, family and neighbours, asking them ‘here, try this’ or ‘whaddaya think?’ is not a usability test. Have an introductory talk with a usability specialist to find out what goes into testing of your user interaction. I am sure you will find it worth your while.

4. if you take UI seriously, budget for it

You are determined to take the necessary steps to give your product the user interaction it deserves. I think the following truism will not surprise you: you get what you pay for. Interaction architects are quite thin on the ground; we are usually busy; and we work at the top of the software development pyramid.

If you want a permanent interaction architect on your team, you are looking at making him or her a partner, with the equity share that goes with it. A person with the right ability—and experience—will know that user interaction is the one and only embodiment of the product, and will only accept a position where the accountability comes with the necessary authority: a ‘C’ position (CUIO?). As the startup grows, more junior interaction designers can be added. But the first one has to be a leader.

Involving external user interaction experts does take a healthy budget. The good news is that if you are familiar with the day rate of a good freelance developer, you will not be knocked out by what interaction firms charge. Bonus tip: the kind of visual designers or front‑end developers who say that they also do interaction design as part of their job, will be exactly the ones who can’t.

5. to save money, you can work more

An important asset of a startup is to compensate a lack of time, money, expertise or personnel by just working more, a lot more. I have used this fact to make a project with one startup happen. It turned out to be very successful and enjoyable. However, there are some hard limits to this approach.

Certain talents and skills need to be available at the startup to be able to take over a good chunk of the menial design and specification work. This is not always the case and it will have to be ascertained before a project starts if sharing the design work is feasible. Then, there is simply no substitute for expertise. Scaling back the involvement of experts simply scales back the depth with which the interaction challenges are met. You get what you pay for.

6. you got to have a vision

Creating good user interaction does not consist of applying some ‘divine’ rules, library patterns or platform guidelines. Also simply copying something that you like from another site or app does not get you anywhere. Designing interaction means creating the embodiment of your product and to help you with that, the interaction architect needs to know what it is.

A product vision is your short, sharp statement of what your product is. It answers three questions: what is the identity of your product; what are the core user groups; where is the end‑user value? The first thing we do in a project is to moderate a session where we help you to draw up your vision statement. Once done, it will be the fundamental basis for answering any interaction design question, our pact for the rest of the project.

If you fail to get your product vision together, then I am afraid your product is doomed. Examples of failing visions are:

• identities solely defined in a derivative way (‘like twitter with music, really’);
• user groups that are impossibly broad (‘everyone’) or schizophrenic (‘for baristas and nuclear scientists, yes’);
• only being able to offer features, when value is asked for.

But not to worry, it is rare for a vision to fail. And when it happens, it tends to be at big corp.

7. zealots, infidels and you

So you got your vision and—as I know startups—you are constantly ‘selling’ it to anybody who wants to listen; including yourself, to keep the faith. But wait, who is actually ‘buying’ it? Well, ultimately only those who invest in it. Either with capital (money, machines, bandwidth, etc.) or with a slice of their life (partners, working ‘for nothing’ for a while). Besides them, there are two more groups.

First there is the huge group of total outsiders to your startup. Talking to them is, at best, a form of market research. From experience I say: because they are not part of creating the future, they are a most reactionary force (‘micro‐blogging in 140 characters? no way’). The magic trick is to stick to your vision, but in the meanwhile filter the occasional nugget of gold out of this stream of feedback.

The second group are those—internal and external—who work with you on realising your vision, creating the future together with you. Are these people cynics, because they refuse to invest in your vision and want to be paid directly for their goods and services? I say: no, these people are very valuable as objective collaborators. Don’t take it as a rejection of your vision, they work with it every day. But without roze‐tinted glasses and no IPO on their mind, they are in a better position to see—and tell you in plain language—the reality of implementing your vision. Make use of their professionalism.

8. you have to build an organisation

Here is something I learned, the hard way: startups have, by nature, no track record of their capability to really develop and ship software. Yes, this is also true if one or two grizzled veterans head up the development. The first reason for this is that the organisation has to establish itself: procedures, communication, support systems, information flow, etc. The second reason is that it takes quite a while to find out if everyone on board is really up to the job.

As a startup you really have to take this into account. It starts with working on that organisation. Yes, even if there is only one developer. There is still information flow; bug tracking; an interaction design to be communicated; perhaps content or a visual design to merge. The one person responsible and accountable for building a software developing and shipping organisation is the CEO, and no one else.

Another way of taking all this into account is to plan with the overhead, delays and other surprises that are caused by both reasons above. With all that agile and iterative development that is in fashion at the moment, ask yourself what the dependencies are if one or two people are not able to deliver. I have seen processes like that completely unravel, with ugly knock‑on effects.

9. structuring is more than half the work

For any project interaction architects do, structuring the design process and implementing our methods is the backbone of our work. The feedback I get indicates that when working with startups, the value of this reaches the next level. I see two reasons for this.

First there is the factor of order. True, for just about any company we work with, our structure and methods supersedes a mostly haphazard and random UI creation process. With established companies, this comes in the form of process change. But with startups this coincides with the organisation building discussed above, and the clean‑cut structure that gets established is much appreciated.

Second factor is clarification. Working with a startup on their product vision; followed by compiling the functionality overview; then some user scenarios to represent typical use; it forces to make concrete what was only there on a subconscious level, or in a state of constant flux. Startups realise that this clarification work we do is a first and necessary step towards shipping an actual product.

10. you will have to manage

As you are building your startup, you are in a rapid process of transitioning from doing everything yourself to hiring professionals—internal and external—to handle all kinds of competences: product, interaction, visual design, development, testing, usability, sales, legal etc. Here is one that is sometimes overlooked: project management.

When you have picked the right people, they bring their skills and industry experience; they structure their work and act with authority—within their competence area; they know what it takes to get it done. But you will have to put them all together. Project management does not just happen by osmosis.

Building that organisation; realistic planning; keeping everybody on‑planning; dealing with contingencies; QA; multi‐platform issues; alpha, beta and rollout phases: you will have to manage it, or have a professional do it for you.

11. ‘real artists ship’

The proof of the pudding is in shipping it. Well, you know what I mean. A product does not exist until it is out on the market. And I mean shipping, not promised, announced or private beta. Now, a quest for perfection gets you nowhere. Question is: how much are you willing to give up to get it shipped?

My point is that you should not be staring at a feature list to find out the minimum you can bare to ship. Instead, think of minimum coherent user experience. If you got to be embarrassed when you ship your first version, let it be caused mostly by ‘missing’ features, instead of ‘good enough’ user interaction.

If you want your‐product‑v1.0 to have an impact and form a basis to grow, then its embodiment, the user interaction, should be designed, not an afterthought. So start small, that helps with getting it out; have the essence of your product designed and then ship it.

12. beware of strangers bearing gifts

There is quite a bit of support out there for your startup. University programmes, incubators, venture partners, business angels, subsidy from city, province, country and the EU. Perversely, some of this support comes with a serious helping of red tape. This can reach a point where the founders of a startup do not have the autonomy to act like executives and entrepreneurs.

I’d swear that the point of a startup is to innovate like no ‘normal’ company can. No naysayers playing it safe, no bureaucracy, no controlling. It means selling the vision to investors, then doing whatever it takes to ship it. And failing fast if the vision or its development is unrealistic.

Startups that live in a bubble, protected from harsh reality by a checks‐and‐balances bureaucracy, end up being humdrum R&D departments. They miss the urgency and agility that enable startups to ship great, innovative products, with great user interaction.

what type of interaction?

And that’s it for now. Maybe you expected something else, like tips about sign‑up forms, touchscreen finger grids, widget toolkits, the merits of toolbars. Actually, the twelve factors discussed above are a couple of magnitudes greater in impact where it comes to user interaction. Some of them are elephants in just about any software company’s rooms. Now what you need to do, is to return to tip one and ask yourself: do I really want to?

Labels: architects, fundamental, practical, product vision, startups

—ps

1 comments, link‐ins, forward this posting:

working on Nokia’s first dual‑SIM phones

3 July 2011, 18:47

I am really proud to see the shipping of the first Nokia dual‑SIM phone: the C2‑00. Additionally a second product on the same basis, the hybrid‐touch C2‑03 (aka C2‑06) has been announced. Having been deeply involved with creating these, I would like to share some of my experiences with you.

revolution!

Although folks on the net write ‘gee, why did it take so long?’, the truth is that developing a dual‑SIM phone is huge undertaking. This is because the software codebase that you start from has been developed—over years, if not ten or more—with the casual, implicit assumption of one user, one SIM.

All of a sudden that fundamental equation is changed to one user, two SIMs. This impacts any software module that deal with voice, messaging or data going over the antenna: i.e. nearly all modules present on a mobile phone, hundreds in total.

a pair is two, no?

And when I say ‘two SIMs,’ I really mean two or more. Dual‑SIM phones do not invent a new way of using phones. They are a response to how single‑SIM phones are used by a large number of users, world‐wide. It is quite normal that these people are using three different SIM cards on any given day. And despite the burden of removing the battery and rebooting the mobile, these SIMs are swapped more times than you would think.

Now imagine what happens to that frequency when one takes that burden away. It was a smart move by Nokia to make one of the SIM slots hot‐swappable.

keeping it personal

When one talks to people about mobile phones for emerging countries, immediately they start speaking of groups, families and villages sharing a single mobile phone. Actually we know what that should be called: a mobile phone booth. Let it be clear that the products I worked on, the C2‑00/03/06, are not designed for that, they are single‐user devices.

Designing the user interaction of a mobile phone booth would be a groundbreaking undertaking. What should Contacts, Messaging or Call logs be like for such a mobile phone booth? I do not know, but certainly not what they are today on all our single‐user phones. If asked, I wouldn’t be afraid to take on the mobile phone booth project, but on the condition that we start from scratch and are supported by a boatload of research to ascertain users’ needs.

structure equals change

Let’s go back to the beginning of the project. ‘One user, two SIMs.’ Changes for dual‑SIM not only uproot the software of a whole phone, but also all user interaction. Same principle: if it has to do—even indirectly—with voice, messaging or data going in or out over the antenna, it is impacted. Yep, just about the whole phone. My job? Making it all work for users.

First thing I did was structure the interaction design part of the project. Having been asked to work as principal designer with a Nokia team of interaction designers, it looked only natural to me that I would design the general solution for dual‑SIM user interaction for the whole phone, and to deliver it in the form of interaction patterns.

The other part of this plan was that I immediately empowered my team of Nokia designers. Each of them was to take charge of a key part of the phone and redesign it, based on the general patterns that I designed. This straightforward plan addressed the questions of how to deal with a project on this grand scale and how to do it with consistency.

nowhere to hide

Also clear on day one was that existing competitor dual‑SIM phones were doing a dreadful job of supporting users with their two, or more, SIM use. To avoid the traps that these had fallen into—identifying SIMs with just a character (A|B or 1|2); always giving up screen space to show/set what SIM to use next—I started with the following principles:

• the solution has to be based on the three factors why users bother to use multiple SIMs: saving money; ensuring network coverage; projecting multiple personalities (life/work, etc.);
• these three factors are not independent—e.g. cost is always an issue; having no network reception spoils the best laid plans;
• these three factors fluctuate over a given day (mobile tariffs, reception depending on location, switching personalities);
• only users can decide on the spot, based on the circumstances and their priorities: is it urgent; can I call later; can I afford to pay more; is this marginal reception sufficient; can I give up my privacy?
• dual‑SIM is not an issue in phone use, most of the time;
• except, when voice, messaging or data is about to go out over the antenna, then users need full control over which SIM is used for that, every time;
• also, when voice, messaging or data starts coming in over the antenna, then users need to identify which SIM is involved, every time;
• make users freely name their SIMs, to express the role each one plays; use these names everywhere;
• design for three‑SIM use, not just the two that are in the hardware slots; also handle if there is just one SIM present;
• keep things nimble; do not lock users into using only one SIM; however, support repetitive use of the same SIM.

the right stuff

The solution I designed ticks all the boxes. In it, dual‑SIM is invisible unless there is a user need to interact with it. Then plenty of screen space is given to display the names given to the SIM cards; to show the network reception for both SIMs, at the moment decisions are taken; for direct control by users, with fast click‐through when everything checks out as expected.

Making it work for users is only half my job. Making it buildable is the other half. My solution avoided that hundreds of screens had to be redesigned. Or that hundreds of UI states had to have new input handling added. This saved a massive amount of effort and cost in design, specification, engineering and testing. Those existing competitor dual‑SIM phones did do all this effort, with sub‑standard interaction to show for it.

The overall architecture of my solution had, of course, a make‐or‐break impact on the feasibility of the whole dual‑SIM project. It is professionally satisfying to see how I was able to align the need for minimal disruption from users, technical architects, developers and interaction designers, enabling the success of the project.

the journey is the reward

After nailing the general solution and interaction patterns, a second phase of working with the Nokia designers started. I mentored, consulted with them and reviewed their detailed designs. Particularly fun was the phase where every week I was in a different country, spending two days on‑site with one of the Nokia designers.

We solved all of the tough interaction design questions during those days. This for instance involved going through all the user settings that relate to messaging and deciding for each if it was impacted by dual‑SIM. There is a surprising number of them and about half require detailed knowledge of messaging protocol or GSM legacy.

Normally nobody bothers with these settings, on either the mobile making or mobile using side. But we had to understand and check all of them. To see, if either for a technical reason—does it stop working?—or to enable any of the three factors, a setting had to be differentiated by SIM. This is the infrastructure aspect of going dual‑SIM: it permeates everything and the team of interaction designers has to think through the whole experience.

more fun

One thing I really want to share is how fulfilling it is to work on a useful product like this for emerging markets. During the project there was a sense, of urgency; of the real impact mobile has on the lives of people in these markets; of how much essential value our designs would deliver to end‑users.

In the same period as I worked on dual‑SIM, the Economist ran a special report on telecoms in emerging markets (make sure you read all six parts). First of all, it is humbling to read that the backwards mobile networks… are the ones in the western world. Second, it is uplifting to read that mobile phones in emerging markets are truly improving customers’ lives—their first phone line, first internet, first means of transferring money—and enable customers to raise their income, which lifts a significant part of them out of poverty.

one more for the road

So that is what it was like. A huge and complex undertaking; fundamental change to the whole mobile phone experience; make‐or‐break responsibility for user interaction and project feasibility; avoiding the traps the competition fell into; structuring, mentoring, travelling, consulting; designing the big picture and the obscure details; and in the end, very fulfilling work.

This is why I am really proud to see the shipping of the first Nokia dual‑SIM phone.

Labels: mentoring, mobile, practical

—ps

4 comments, link‐ins, forward this posting: