publications

GIMP redux, full GEGL ahead

22 January 2012, 17:36

Recently I have been receiving requests to clarify the GIMP UI strategy surrounding GEGL, so I thought I’d write a catch‑up blogpost about my 2010 LGM lecture. There I tackled this GIMP UI challenge: a first outline for a UI for a fully GEGLed GIMP. The thinking about this UI, and the discussions with Øyvind Kolås (the GEGL‐meister himself), have been going on for years. Its introduction will be the most profound change to GIMP in the foreseeable future.

two minutes of fame

I started off my lecture with introducing GEGL, the graph‐based image processing engine that is slowly, but surely, being integrated into GIMP. I could spend quite some time taking us through the complete GEGL graph (linked) below:

…and this is only a small one. Instead, here is the essence of it in four easy steps:

1. there are input boxes that load an existing image, or render some vector shapes or text;
2. there are chains of operation boxes that perform things like blur or change opacity of whatever gets fed into them;
3. there are composer boxes that take two inputs and put one over the other, combining them in a certain way (think layer modes: normal, dodge, multiply);
4. there are output boxes that either display the grand result on a screen, or export it to graphics file formats like png or jpeg.

so…great

Thus GEGL processes graphics by hooking up the boxes, inputs–to–output. Why does that matter? Well, because it is non‐destructive: the images in the input boxes are never modified.

If the structure of above graph is written to a file—apart from the input images, all other boxes are just snippets of XML—and a year later it is re‑opened in GIMP, then each of the operations and their parameters; each of the vector shapes or text can be freely changed. Even the input images could be swapped out for different ones. The result is a changed image composition, without any loss of quality.

It is exactly this promise of non‐destructive editing that played a big part in me joining the GIMP project years ago. I could see how that could lead to the end of some of the major workflow bottlenecks in today’s graphics software.

UI modelling

The integration of GEGL in GIMP is a disruption; it changes the rules of what one can do and how one can work. This is not a bad thing, it is a refreshing change. An interesting question is: how does the user interaction of GIMP have to change, in order to harnesses all this new power? In general there is a big urge, especially with developers, to just display the graph on the screen:

I call this the boxes and hoses model. If it looks familiar to you, it is because it has been around for decades: it is called visual programming. Which again explains why developers tend to choose this type of solution. One day a direct representation of the graph will appear in GIMP, as something extra. This is because the product vision defines GIMP as (also) being ‘a platform for programming cutting-edge image processing algorithms, by scientists and artists.‘

activity centre

To find out what the main UI in GIMP should be, we take from the product vision what the main activities are that GIMP is made for: ‘high-end photo manipulation’; ‘creating original art from images’; ‘producing icons, graphical elements of web pages and art for user interface elements.’

As a next step, it pays off to look at the nature of these activities. Users start with images or basic shapes (vectors) and apply image manipulation operations, one after another, to achieve the desired result. Users organise their work in layers and GIMP composites the result. Schematically that looks like this:

In short: a work (a composition) consists of layers, each with its sequence (in time) of operations. Now we got the start of a model for the UI. That list of layers we know already, as the layers dialog. The sequence of operations for the layer, we can call that the operations dialog:

disclaimer

Keep in mind that the user interaction shown in the image above, and all the ones below, is not a true mockup. It is more a diagram—with in part grotesque proportions—to show the principles of how the UI works.

We can see that the four GEGL elements are covered: The image material to start with: loaded pixels or rendered vectors; the layers that control the bulk of the compositing; the chains of operations; the output to the screen in the big image window.

Yes, the image window is the place for judging one’s work and for doing the actual work, hands‑on. The image window deserves a couple of times more screen space than any GEGL graph manipulation. Reversing this relationship (as shown earlier) is completely disregarding the nature of the activity.

back to the future

Adding an operation works as before: use a toolbox tool or invoke a menubar item. Here Gaussian blur is invoked and it appears at the top of the operations list:

Now it gets more interesting: revisiting the past. No matter if it was done five seconds or five months ago, one can recall any previous operation applied to this layer and readjust it. Here the curves for this layer are revisited. While adjusting, the output updates through the complete chain of operations, including colorise and blur:

A lot of layers dialog behaviour can be reused for the operations dialog. ‘Eye’ toggles can be used to show/hide the operations:

Drag and drop can be used to rearrange the order of operations. The result of that may turn out to be more subtle than you think. Because GEGL processes everything in 32bit floating point, it will be much harder to get the rounding and clipping artefacts you get with 8‐ or 16‑bit integer processing. Only when an operation does not commutate by nature (i.e. the order matters), then it gets interesting to experiment with the order of operations.

Of course copy and paste of operations, between layers of the same or different files works. Dragging and dropping of operations on another layer copies or moves them, a modifier key (shift or ctrl) will sort that out. And before I forget: selecting some operations in the operations dialog and invoking ‘Make macro’ would be a good, natural way to get that started.

the holy trinity

Before we continue with more GEGL user interaction, here is a pop quiz. I took this b+w photo of the brilliantly named Kloten airport and coloured part of it red:

My question is: how did I apply that red?

1. directly with a tool from the toolbox;
2. making a selection, then invoking a menu item;
3. using a layer with a layer mask.

Well, I have forgotten how I did it (it’s been a while) and you will never guess. My point is that it does not matter. All three methods listed above are a combination of an operation (apply red) and a greyscale mask, controlling where it is applied and how much. All three methods are fully equivalent, it does not matter to the software—or the viewer.

It does matter to users. Given the context, composition structure, the graphics material and the end‑goal, each user knows exactly which of the three methods she/he prefers. There is a million different use cases and only the individual user on the spot can take the right decision. It follows that each of the three methods is equally important and needs to be equally available to users. This I call the holy trinity of image manipulation.

pagan practices

However, at the moment, you quite often see the following: ‘if you want this feature, you’ll have to use it on its own, extra layer.’ This is layer abuse. I get misquoted on this so let me clarify: users never abuse layers, developers do. Here are some examples of layer abuse:

• the only way to do a non‐destructive operation is via an adjustment layer
• only one vector shape per vector layer;
• only one block of text on a text layer;
• the output of a filter plugin is always put on a new layer;
• the result of using a toolbox tool is always put on a new layer.

The problem is with ‘only,’ ‘always’ and ever more layers, whether users want them or not.

reformation

The abuse listed above is straightforward to fix. Quite a bit of it has to do with enabling users to redo or revisit the image manipulation. That is solved by the operations dialog.

Furthermore, there can be as many vector shapes and text blocks on a layer as one likes. Just show them—and stack ’em—as sub‑layer elements in the layers dialog. And when then one of these sub‑layer elements is allowed to be actual pixels, then it is clear that the whole notion of special vector/text layer can disappear:

Layer abuse has to stop. Developers should never force users to use another layer. Only users decide how many layers they want to use, purely as their own personal way to organise their work.

more blessings

So apart from that no image manipulation is exclusive to a layer, what are further implications of the dogma of the holy trinity?

paint with anything

Yep, with anything, also all the plugins that appear in the Filters menu. Also the obscure ones that you and I, and anyone on the GIMP team have never heard of. Just dip your brush in it and smear it on the canvas. Thinking of combining it with paint dynamics just blows my mind. It just takes one tool (Filter brush?) to enable this.

adjustment layers with anything

You see, I have nothing against accomplishing things with layers. It is perfect when you got a collage made out of dozens of layers and then want to apply some treatments to the whole thing. It just takes one adjustment layer with any number of operations to get that done. Again, with anything.

behind the mask

As mentioned, all three image manipulation methods consist of an operation and greyscale mask, controlling where it is applied and how much. The interaction for these masks can be analogous to that of layer masks:

This means that both the parameters of the operation and also the ‘where and how much’ can be revisited and adjusted, a second later or a year later.

in triplicate

By now you may be thinking ‘wow, every image manipulation must be possible in three different ways, does that mean that one day GIMP will contain three times more stuff than today?’ Well no. I have already mentioned two measures—Filter brush, adjustment layers with anything—that will make large strides towards fulfilling the holy trinity. And even today the trinity is partially in place at GIMP.

Take for instance composition, putting pixels on top of pixels. Sounds like the exclusive domain of the layer stack, no? Well, even today you can use toolbox tools to do the same thing: the Clone tool and its 90%‑identical cousin, the Heal tool. And the menu item to compose is called Paste. Yes, the current layers dialog interaction while pasting should go. The GIMP team is agreed on this for years. Instead paste is an operation on the current layer:

weird science

There is a fifth element to GEGL graphs, that I have been keeping under wraps in this blogpost until now: cloning. This is taking an in‑between result in the GEGL graph, anything from a simple vector to a huge sub‐tree, and ‘teleporting’ one or more clones of this result to another position in the graph:

The original and the clones can have the same or a different position on the canvas; undergo the same or completely different operations; be composited in the same or a completely different manner. The magic is of course in that when the original (the input to cloning) is updated, every clone immediately updates, with all further operations and compositing applied on top. This is going to be liberating, because of the amount of work this saves.

Let me give an example. Let’s take the scan of a snowflake and fix it up with a number of graphics operations. We clone the result 99 times and spread these over five layers to compose our snowscape. We make ever flake a different size and a different colour. Already laborious, no? And now, we decide we are not satisfied with how refined the snowflakes are. Solution: fix the original with more/less/different/updated operations, all 100 flakes will immediately show the improved refinement—in their different sizes and colours.

send in the clones

My plan for the user interaction of cloning is to make it a variant of pasting: Paste as clone. This includes doing further operations on the pasted (as clone) material, before closing off the paste (you can always revisit the paste operation and modify all of it). And as Ville Pätsi pointed out a while ago, there needs to be a way from each clone to access the original. A button or link on the Paste clone operation in the operations dialog might do the trick.

There is more tricky stuff coming up with this cloning, like cloning layers or layer groups. And to go fully meta on this: I am asking myself why chains of operations cannot be cloned and applied to different input material. But Øyvind is not up for that.

two more things

To wrap all this up I have two benchmarks that have developed out of the GEGL discussions. They are alternate manifestations of the holy trinity and they are serious tests of the direction the GIMP user interaction is taking.

First, in a future version of GIMP, if users really envision their work as one single layer—not that unusual with photographs—then they will simply be able to close the layers dialog. Their work will not be impeded in any way by this, everything is still available to them (twice, as toolbox tool and menu) with no limits in the sophistication with which they can express themselves:

Second, in a future version of GIMP, if users are really not interested in switching operations on/off; rearranging operations or revisiting the past, then they will simply be able to close the operations dialog. Their work will not be impeded in any way by this, everything is still available to them with no limits in the sophistication with which they can express themselves:

You could even combine the two benchmarks and GIMP would still work, without a hitch.

aftermath

Directly after the lecture at the LGM, Øyvind Kolås pointed out one flaw, one thing I had not thought of: those greyscale masks, that play such a central role, are not pixmaps. They are GEGL sub‐graphs themselves. Which creates the possibility of near‐endless drill‐down: operations with greyscale masks, which contain operations with greyscale masks, which contain… et cetera.

Is that a fatal flaw? Is that going to blow up my plan to reduce the complexity of the GEGL graph to a linear list of operations? Do we just have to confront users with the whole graph? Let not kid ourselves about the complexity of that graph. In practice, I’d say that a simple GIMP file starts at ten times the number of boxes in the example at the top, quickly moving to a hundred times and then beyond.

not my problem

To developers—especially the ones that are working up to their elbows in GEGL graphs—this problem looks like one of navigating the graph and making edits. But to me it looks like another problem needs to be solved: how can users navigate their working context? Already today GIMP users are doing this navigation. Which layer, which layer mask are they editing, or is it the quick mask?

It will have to wait for another day, but when this context navigation is going to be designed, the one thing not to lose sight of is the image window. Everything evolves around it, including this navigation, and feedback of the working context has to be inside it. I certainly think we can do a better job than currently happens with the layer masks.

And when this context navigation has been designed, taking into account the holy trinity and the two benchmarks, then the operations dialog can show the simple list of operations performed in that context, in their natural order.

Labels: GIMP, GIMP redux, lecture, practical, product vision

—ps

30 comments, link‐ins, forward this posting:

teaching interaction /10

18 August 2011, 18:16

This is a catch‑up blog post, about the 2010 edition of my course, interaction design for the real world, at the FH Vorarlberg, Austria. Last year the course fell right in the middle of my involuntary blogging sabbatical. However, everyone involved—students, FHV, the GIMP project—deserves that the results are published, so here we are.

For developments in the goals and structure of the course I will refer to this year’s and previous instalments. So let’s cut straight to the chase, the class‐of‐2010 interaction design challenge.

natural selection

For a couple of years, I have had the notion that the four tools in the GIMP toolbox that perform selections based on pixel values are quite similar. Select by color ‘works a lot like the Fuzzy select tool.’ Intelligent scissors traces ‘any high‐contrast edges it can find.’ Foreground select appears to do all of this, on steroids.

from left: fuzzy select, select by color, intelligent scissors and foreground select tools

It could be worthwhile to condense these four tools into one, maybe two, new ones. This meshes well with my top‑level interaction architectural goal of reducing the number of tools in the toolbox. Thus it became the students’ brief, a top‑down goal driven by design intuition.

Since this type of project is designer‐initiated—instead of client‐driven—there is more creative freedom involved, to define further goals and even to pick the right amount of functionality.

use the force

All that freedom might look like a recipe for artistic doodling and ‘design’ for its own sake. Luckily methodical working—one of the cornerstones of the course—ensures that also these type of projects are properly researched and analysed, leading to rational and practical results.

The guiding rails for the evaluation of the existing software and new designs were, as usual, the user scenarios. We constructed three. The first one embodied that simple things shall be simple: selecting a square of uniform color (rotated so that the rectangular selection tool would not work). This should be a no‑brainer with the redesigned selection tool(s).

A second more complex scenario was selecting a car, separating it from a real‐world background. This meant joining the several parts that make up the car into one selection. The third dealt with selection of material spread over several layers.

three team results

And now the design results of the student teams. The work of all three teams is available under the GNU Free Documentation License 1.2. I present them in team order.

team one

• members: Martina Peer + Adriàn Fallas Valverde
• design concept document (pdf, 194 KB)

Like the other two teams, team one decided on a single selection tool to replace the four old ones. They boiled foreground (content) selection down to the essential: drag the mouse over what has to be inside the selection.

Complementing this stark reduction, team one thought outside the box and combined the free selection and straight line aspects of the free‐poly tool, with the edge‐finding capabilities of the intelligent scissors:

team two

• members: Marianne Minar, Sebastian Platzer + Florian Steinbichl
• design concept document (pdf, 2.8 MB)

Team two used a gesture (‘shake’) to engage foreground selection mode; it matches the wavy lines one typically sees drawn when marking the foreground object:

Shortcuts were handled deftly by team two, with <shift> always having the meaning of addition and subtraction engaged by <ctrl>.

Promising is the natural way how manual adjustment of the selection outline was integrated by team two: approaching the selection outline with the mouse pointer reveals path nodes automatically inserted for this purpose; nodes can also be added and deleted:

team three

• members: Steffen Fridgen + Sarah Montti
• design concept document (pdf, 6.7 MB)

Team three designed a pie menu to switch tool modes, it appears under the mouse pointer when <shift>‑x is held:

A good design move by team three was to take some of the more deluxe aspects of the old foreground select tool and make them generally available in the new tool. For instance, the exclusion of certain parts of the image from the selection algorithm could be set up with the Area mode (shown with its tool options):

Team three finished their documentation with a clear flow chart of how selections, both simple and complex, can be accomplished using their design:

back at the ranch

I would like to thank the students of the class‐of‐2010 for their hard work. My apologies for taking this long to publish the results. With these dues paid, I can now look forward to teaching interaction, in 2012.

Labels: GIMP, teaching

—ps

0 comments, link‐ins, forward this posting:

teaching interaction /11

27 July 2011, 23:18

A few weeks ago I was again teaching my course, interaction design for the real world, at the FH Vorarlberg, Austria. The goals and structure of the course can be found in previous instalments. This year, in the name of continuous improvement, I made one change to stamp out what had been bothering me in previous years.

With alarming frequency, my students had been writing in their reports that one of the design goals was ‘to create an intuitive tool for GIMP.’ As I have written here before, there is no such thing as intuitive: all interaction is learned. All an interaction architect can do is harmonise the learning curve of a piece of software with its other design goals.

So this year I paid special attention to the fact that GIMP’s product vision sets ambitious goals for the freedom to create and for efficient working in production environments. The unavoidable trade‑off is that some serious training and practice is required from users before they can unlock this freedom and efficiency. I am happy to report that my students really tuned into this vibe and got their design priorities the right way around.

GIMP warped

Another year, another real GIMP design challenge for the students to work on. This time I picked the current summer of code project to bring the iWarp plugin to the toolbox.

iWarp allows to deform image content by pushing pixels around like putty; growing or shrinking areas; swirling in either direction and touching up all of this by smoothing or removing these deformations. Taking this out of the confinement of a plugin and making it a true toolbox tool has been an obvious goal to the GIMP team for years. That seems to be a straightforward project and automatically solve quite a few problems: undo, zooming, panning and selections become available ‘for free.’

no free lunch

As the students found out, things are not that simple. Delivering the aforementioned freedom and efficiency asks for some serious interaction design. This multiplies with the fact that for a toolbox tool, opportunities for shortcuts are sparse: just about all keyboard shortcuts are already taken by the tool switching system and menus.

Another challenge was the integration of the pressure input of a graphics tablet controlling the radius and/or amount of the deformation. There were lots of design choices to be made about how straightforward or powerful one wants to make such a system.

scenarios + evaluation

To facilitate evaluation, we set up on the first day of the course three user scenarios that map out ‘the playing field’ of using iWarp. They ranged from perfecting realistic photos to wild ‘action painting’ with the deformation tool. In the middle of that range is the scenario in which a fantasy world is created out of a photo.

During the expert evaluation phase we not only evaluated iWarp on its own, but it was also quite useful to perform competitive analysis of photoshops’s equivalent tool: liquify. I was surprised to find that it is also a boxed‑in plugin, with a lot of functionality added to compensate for not being a toolbox tool, e.g. custom masking to compensate for not having the selection system available.

When done right, the in‑the‐toolbox strategy is clearly superior, in a way that a plugin can never compensate for. Also during the evaluation the students noted that iWarp was noticeable snappier than liquify, the latter bordering on being too slow to match the goals we had set for keeping out of the way of users’ flow. I can only hope/insist that iWarp maintains that advantage.

four team results

And now the design results of the student teams. First let me make clear that the work of all four teams is available under the GNU Free Documentation License 1.2. I present them in team order.

team one

• members: Selen Gürün, Recep Emre Koca + Sandra Beckstein
• design concept document (pdf, 1.9 MB)

Apart from the freedom and efficiency goals, all teams were asked if there really are seven distinct deform modes (move, grow, shrink, swirl CW, swirl CCW, smooth and remove).

Team one took grow–shrink and swirl CW–CCW as positive–negative instances of the same deformation (resize and swirl). Uniquely, they put these on the ends of the Amount parameter scale. Thus, a negative amount does the opposite of a positive amount and (near) zero effect is in the middle of the scale:

Team one elegantly managed what was a challenge for all the teams: the combinations of static and/or dynamic (by means of pressure from a tablet) control of radius and amount. They designed a clear dynamic override for each parameter and a full curve system for each, including reverse effects (more pressure, less radius or amount):

Finally, team one did a good job argumenting their design in their document: like it should be done, assertively, relating decisions back to the client’s product vision and the design goals.

team two

• members: Ingrid De León + Stephanie Nagel
• design concept document (pdf, 2.5 MB)

During the brainstorming phase, team two surprised me with a mind map of the problem and solution space:

Team two gave every deform mode—combining the swirls—its own tab to express that the parameters were unique for each mode. Then in the seventh tab (bottom one) they put the pressure dynamics handling:

team three

• members: Lorena Chan Guerrero, Niek Gerrits + Jennifer Paul
• design concept document (pdf, 3.3 MB)

Team three bristled with creativity and worked fast. I kept them on their toes by giving them one next‐level design challenge after another. First, the team designed a good set of gestures and a system for employing them to switch the deform mode of the iWarp tool (as a shortcut, of course, using the mouse right‑click):

Then, team three went furthest in combining deform modes. The beauty of this is in the shortcuts they attached to them, with <shift> toggling between the left–right columns in the picture below and <ctrl> setting the mode to detail while pressed (i.e. <ctrl> engages smooth for a moment and <ctrl><shift> erase):

Furthermore, team three came up with two‑dimensional input for the radius and amount parameters which has some really interesting workflow potential:

team four

• members: Heike Marlene Rosenberg + Analucia Moreno Concha
• design concept document (pdf, 4.7 MB)

Team four had their own take on how to integrate the pressure dynamics into the parameters. The simple parameter slider (mouse view, below) for radius or amount changes to a dual slider (tablet view) which sets the parameter span between minimum and maximum pressure. Simple, effective solutions like this are not to be underestimated:

What was also a good move by team four is that they stepped in to deal with an issue that they found during their expert evaluation: how to deal with pixels on the edge of the region being deformed. This looks like interaction architects crossing over into the territory of pure algorithm design. However, since user expectations and interaction behaviour is involved, it was the right decision to start getting involved:

postscript

Teaching at the FH in Dornbirn is an annual highlight on my calendar and also this year did not disappoint. The classes are demanding for everyone involved, including me, but already at the end of the first day I was reminded of how much I enjoy it. Thanks to all the students for the hard work they put in and for the energy they returned.

Labels: GIMP, teaching

—ps

0 comments, link‐ins, forward this posting:

working on a vision with Krita

5 March 2010, 16:42

Last weekend I was at the Krita sprint, to work with them on defining their product vision. And hey, while I was there anyway, I could also help them with some UI first aid. Krita could simply not continue without having a vision. Without a clear definition of what they are trying to achieve, the project was threatening to go nowhere, at an ever-slower pace.

british invasion

Now, I would hate to see that happen. Similar to the Beatles and the Stones, the animosity between free/libre graphics projects is blown out of proportion by the fans. The Jagger, Richards, Lennon and McCartneys of GIMP and Krita get along just fine.

GIMP definitely does not try to be the graphics application for everyone. So there is plenty of room for other applications to address user needs that GIMP does not want to cover. And although GIMP, Krita and other apps may have quite a bit overlapping functionality, their product visions may be different enough to prevent them from being direct competitors.

Even then, I would not have been upset if a large overlap with GIMP had been the result of Krita’s vision. I was simply curious to find out what it was. So I offered my services and got to play Mr. Wolf for a weekend.

life is a beach

These days I have have some criteria for doing pro bono jobs like this. Is it going to be fun? Is my work going to be appreciated? Am I going to work with some great people? Do I look forward to the travel and the location? It better be good. The Krita sprint ticked all the boxes.

Hospitality was great and the location could not be farther opposite to ‘air conditioned conference room at the airport hotel.’ Working in the living room of an old house in the middle of the historical town centre is a treat. I had never been in Deventer, but it turned out to be almost an open‑air museum of a cute, old Dutch town.

hashing it out

Our vision session was on Saturday morning and it felt very much like when I started working with GIMP: OK, it’s a graphics application but what are they trying to achieve? The impact of this was demonstrated the night before, when the Krita team was discussing some of their UI problems and I realised that without a product vision I simply was not able to help them in a meaningful way.

So on that morning we sat down and I explained to them that I was there as the interaction consultant and not as the‐man‐from‐GIMP; that I always do this method at the beginning of all my projects, starting as a complete outsider; that it is their vision, because Krita is their life; but also that their vision has to be formulated through the eyes of users (they don’t care that Krita is written in C++, for instance).

the knockout

Then I set them off with the three magic questions—what is it, who is it for and where is the value?—and moderated the session from there on. I made sure that we got definite answers on all the questions, even when having to make a choice was no fun. I also made them realise how much hard work there was involved with stating to support certain user needs.

Before I travelled to Deventer I was both anxious and curious about the Krita–KOffice situation: a paint app with natural brushes as part off an office suite? The discussion of this took its time, stretching our session beyond lunch.

Essentially, all I had to do was show them that there were only two options (make sense of Krita being part of KOffice, or leave) and insist that they did not dodge the question. It turned out that Krita was already on a path of severing ties with KOffice and we reflected that in the vision by not mentioning the latter at all.

putting it all together

After the session I wrote up the vision, short and sharp, I like it that way, so it can be used as a tool in the project. After useful feedback by the Krita team and a final adjustment, it was ready for prime time:

‘Krita is a KDE program for sketching and painting, offering an end–to–end solution for creating digital painting files from scratch by masters.

Fields of painting that Krita explicitly supports are concept art, creation of comics and textures for rendering.

Modelled on existing real‐world painting materials and workflows, Krita supports creative working by getting out of the way and with snappy response.’

the Krita product vision, as written by ps

I am really happy how coherent their vision turned out to be. Funny enough, there is about zero overlap with the product vision of GIMP.

Krita’s maintainer, Boudewijn Rempt, did an excellent write‑up of the far reaching implications of this compact vision. After he published it, he remarked that he could feel how carefully the words had been chosen and placed by me. Yes: nothing is a coincidence in there, it all fits together like an intricate puzzle.

instant karma

The product vision was immediately placed on the Krita website, signalling what was already going on at the sprint: thinking, designing and working in a new, clear context.

Suddenly there was a clear basis for deciding which plugins should be removed from the standard distribution and moved to a online repository of user‐installed options. The vector tools are receiving a hard look at the moment, now that the focus is so solidly on painting from scratch.

the Bonnie situation

With a product vision, I was able to help them with many a UI topic. We worked on the pop‑up palette; brush engine parameter input; the overall activity of setting up a brush, just right; brush presets, managing and using them; a strategy for dockers and dialogs on screens small and large.

I brainstormed UI ideas; rejected many bad UI solutions from a passel of other paint and pixel apps; made concrete solutions; outlined direction for big issues; connected the dots between related UI issues; concentrated on what it should feel like to work with Krita; I showed how even within the compact scope of the vision, plenty of diverse use has to be supported.

One thing I emphasised was to move away from ‘how would I like it’/input of a few users/looking at other apps for UI solutions. That is not the way to create UI, I’m afraid. Instead, the Krita guys should build up an understanding of the activity of many users that fit their vision and always make solutions for a hundred thousand of them.

different strokes

It was interesting to see for me that solutions that I am working at for GIMP would not be fitting for Krita. This is not only a matter of having different visions—although it suffices. I noticed that also the different as‑is situation creates different context and leads to different UI solutions.

Wrapping up, I want to thank the Krita team for the great cooperation during the sprint weekend. At times it was quite intense, but always in a positive atmosphere. It is very gratifying for me to see in Boudewijn’s and Cyrille’s blog how they are charging ahead with their new‐found élan.

Labels: GIMP, practical, product vision

—ps

3 comments, link‐ins, forward this posting:

from open source to symbian

3 November 2009, 19:42

Last week I was in London, at the symbian foundation and SEE2009. The reason for that is that I am a complete schizo. Well OK, not really. Let me explain: there are these two major—but completely separate—themes in my career as an interaction architect.

First one is mobile. For more than twelve years I have been working on fundamental and application interaction design for mobiles. There are more than half a billion mobiles around with software that I designed.

These days at m+mi works, I am fortunate to be lead interaction architect on some of the more interesting mobile projects, solving their fundamental challenges; I enjoy mentoring a next generation of mobile interaction designers for our clients; and I provide deep analysis of mobile usability test results, showing cause and solutions.

the other peter…

The second theme is open source. Since 2005 I am involved in the open‐usability scene in Berlin. You can read all about my involvement with openPrinting and GIMP on this blog. However, there is no mobile in my open source work and no open source in my mobile work. That’s the schizo split. It was because of GIMP that I ended up in London.

When Scott Weiss, UI technology manager at the symbian foundation, adopted our UI brainstorm to start the symbian UI brainstorm, I got in touch with him. A good talk resulted in an invitation to do a presentation during a symbian foundation UI workshop and to take part in a panel at SEE2009. Both with the aim of sharing my experience in working with open source projects with key players in the mobile industry.

fast forward

The condensed version is that commercial and open source work are 90% the same. Which means that everything learned in commercial projects can be used in open source. And vice versa, new methods I developed for open source projects get used in my commercial work.

The 10% difference is openness —surprise!

This openness creates an interesting dynamic where users are literally using today what was programmed yesterday. And that for serious work. This allows for experimental modes of interaction design and creates unique opportunities for usability testing.

However, as I have explained before: without usability or interaction design experts on the project, the very direct contact and feedback between users and developers in open source only leads to awful usability.

seriously

Working on open source is not just a hobby of mine. It is a strategic activity for m+mi works. By working openly and publishing about it in this blog, we have a fighting chance of making understandable what interaction architecture is. And making clear that it is missing in most development processes today.

You will notice that this blog does not talk about all that mobile work that we do. All of it is under NDA. Which is logical—in a closed source way—because yes: interaction architecture is highly strategic. It fully determines what a product or piece of software is, for its users.

feels like yesterday

As part of my presentation, I recounted how my involvement with GIMP started; talked about methods like the scenario weekend; gave examples of ideal cooperation based on infectious enthusiasm; and of course told the story of the making of the brainstorm.

At the SEE2009 panel, form, audience and the venue were different, but above condensed version was woven into the narrative of the panel. Both the workshop and SEE2009 turned out to be good opportunities to meet interesting people. I thank Scott for inviting me to be part of it.

More related musings in a couple of days.

Labels: fundamental, GIMP, lecture, mentoring, mobile, practical

—ps

1 comments, link‐ins, forward this posting:

GIMP redux, single‐window mode

18 September 2009, 19:19

This blog entry covers the final part of my talk at the libre graphics meeting this year, after dealing with the schmuck and squaring a circle. It will also be pure falsification of history, Soviet‐style: I used my talk to kick off a discussion and to start the design process. Most of the interesting things I am showing you today were not even conceived when I delivered my lecture.

real history

Here is what was presented at LGM: First, I set the mood by showing a selection of single‐window contributions from the UI brainstorm:

The high number of contributions in the single‐window category confirms that this is a major issue where action needs to be taken. The many forms the contributions take shows us that a measure of flexibility and configurability is needed.

Next, I reminded everyone that it is a fifty–fifty world: about 50% of GIMP users love the multi‐window interface of today and do not want to lose it, about 50% of GIMP users would love to move to a single‐window interface.

one switch

With the world fifty–fifty split, we will serve them with one switch in the Windows menu, switching between multi‐and single‐window mode. I used the mock‑up with the beautiful orange sports car from the brainstorm to demonstrate the basic idea (no real UI design implied here).

When set to single‐window mode, each document will appear in a tab:

the toolbox and inspectors can be ‘torn off’:

here the toolbox is side‐docked back and it is shown how the inspector floats:

and last, inspectors can be made multi‐column. Here, another column has been created to allow more dockables to be organised:

The eagle‐eyed will see that there is quite a bit of fudging in those images, so don’t sweat the details.

three problems

I closed off the talk with three things that were bugging me at the time:

no tab tear‐off

tearing off a tab and making that into a separate window—as seen on web browsers—would make our single‐window interface into a multi‐window interface. How annoyingly inconsistent.

no side‐docking, to multi‐windows

both toolbox and inspectors are unique for the application and by nature there are multiple window instances. That means it is unnatural to dock the toolbox and inspectors to windows. It is however possible to introduce multi‐column inspectors for multi‐window mode.

no image comparison

With tabs, only one image is visible at the time. Thinking about clever plans to see two images side‐by‐side in split panes, one sees immediately the confusion rising: which is the image that the menus perform operations on. Which one is shown in the layers dialog?

tomorrow starts now

So far, so predictable. Here is where the interesting stuff starts. After my talk I had a good round of brainstorming with Martin Nordholts on the tear‐off and comparison problems. The result of that is what I call the polaroid.

To demonstrate it, let’s say one is still working on that lovely orange sports car and in a second tab we have loaded this image of a cool Ferrari Berlina:

Now we want to compare our orange creation to the Ferrari. All we have to do is to tear off the Ferrari tab and presto: a polaroid of it appears:

This is not a GIMP image window. It is not even a window, the window manager will know nothing about it. We avoid making it look like a window, down to ignoring the theme colors and executing the control strip at the bottom in paper white. You can move and resize the polaroid to your heart’s content, as long as it fits inside the GIMP window. You can zoom, pan and look at it, and that’s all.

voluntary confinement

That last point will be very hard to do, because everybody will be thinking of one little feature they would like to add, for themselves. But we have to be strict here to avoid mass confusion: every toolbox tool, every menu item, every dialog (in an inspector) only operates on the front tab—the orange car in this case— and never on a polaroid.

For example, when I discussed these plans with GIMP‐usability contributor Ellen Reitmayr, she asked: ‘so can you then select and copy a bit of image from these [polaroids]?’ Good question. After realising that this would mean interacting with the layer stack and selection tools, the answer has to be no.

It is however possible to open multiple polaroids of the same image. And also of the image in the front tab, where one is working on:

As you can see they can overlap and they will have to have their own stacking order. Just click on a polaroid to raise it to the top. The final thing you can do with a polaroid, is to close it…

the decline and fall of tabs

This is how things were more or less when I returned home from LGM. At that point there were still three things that were bugging me about using tabs in single‐window mode:

1. too easy, too fast: the tabs got adopted from web browsers, without much thought;
2. it would be rather logical to use a thumbnail on the tab itself to identify the image;
3. sign of the times: the web browser folks themselves are looking at solutions beyond tabbed browsing.

And pretty soon I was looking at the image parade you see these days in many a photo app and viewer, mainly triggered by a visual association with a row of tabs with thumbnails on them:

There are some really interesting possibilities, when we replace tabs with that. Let’s try it out:

We see three open images and like you would expect, clicking on an image makes it the active image (the ‘front tab’) and dragging any of them up or down enough ‘tears’ it off to become a polaroid. A mouse‐over tooltip will display more info for each thumbnail, for starters: the filename.

history repeating

Now let us go a step further and not only show open images in the parade but also recent files that have been opened or worked on, the history:

Everything is in strict chronological order: open a file and it moves to the front of the parade. What I find very intriguing is that the notion of which file is open starts to blur. Having a file being loaded from disk becomes a side effect of clicking it to make it the current file. Similarly GIMP can decide to start closing files with no unsaved changes when there are too many of them open and memory gets tight.

My current thinking is that it is not necessary to mark files in the parade as being open. The concept quasi disappears for users. The white stars indicate unsaved changes.

One or more graphics files that are dropped on the parade get added—cued up if you want—right after the open files. Just as useful will be that a folder dropped on the parade cues up all the graphics files in it. Current work + history + cued up work + polaroids has a lot of potential, I think.

the Midas touch

I realised that it would be an eyesore and an absolute waste of space to introduce a scrollbar for the image parade. Still, it is going to scroll. It happens to be that in the last couple of years I have been working for two large companies on the interaction fundamental’s of iPhone competitors.

So how about using touch UI? A flick or a drag along the parade scrolls it; a push slows down, then stops the scrolling; a tap on a thumbnail makes it the current file; a flick or a drag of a thumbnail perpendicular to the parade creates a polaroid: it is perfectly doable, using mouse, trackpad or tablet.

non‐preferential customisation

All we need now, is some more flexibility. First, I think everybody will have their own idea what the right size for the image parade is. From pretty small:

to pretty big:

I think any size in between those has its purpose and why could that not depend on what one is doing next? Simply grab the dividing line between parade and canvas and adjust to fit your mood. Next, does it has to be at the top of our window? Left, right and bottom should also work:

I foresee every GIMP user having their own sweet spots, when it comes to combinations of size and orientation of the image parade for the type of projects they do.

raising the dead

It will not be necessary to introduce new items in the preferences dialog for anything I have shown you here. In that light I called during my LGM talk preferences ‘the graveyard of many a good idea.’

For configuring the image parade I have already built in a settings pop‑up in all the screens you see above. The top/left/right/bottom setting will be in there, as will be one for controlling the amount of history. Something along the lines of: 48 hours; a week; a month. And for those who never look back: a setting for no history at all.

red hot anticipation

To wrap up, for about 50% of GIMP users the good news is that single‐window mode is coming, complete with tear‑off toolbox and inspectors; multi‐column inspectors; polaroids; image parade with history and cued up work; touch UI and quick and easy customisation.

And it is coming faster than you’d think. Martin is ready to develop it for version 2.8 and this blog post was hotly anticipated to outline my complete plan.

For a different 50% of GIMP users the good news is that multi‐window mode is here to stay. It will benefit from multi‐column inspectors being introduced. But I actually expect that once single‐window mode is out there, there will be pressure from the community to have a look if multi‐window mode cannot be done a bit more, well, modern.

Labels: GIMP, GIMP redux, lecture, practical

—ps

82 comments, link‐ins, forward this posting:

teaching interaction /09

28 July 2009, 14:40

At the end of May it was again my pleasure to teach interaction design at the FH Vorarlberg, Austria. I will discuss what was new and notable compared to last time, show which new GIMP functionality we worked on and then present the results.

full intent

Kicking off with ‘what’s different?’, I had fine‐tuned the goals of my course:

structure, structure, structure

If there is one thing I want my students to take away from this course it is a project structure that enables them to tackle any (interaction) design project. I took the structure and methods used in my daily practice and condensed them to a project survival guide. In a fine example of ‘the medium is the message,’ these structure and methods also form the programme of the course.

true interaction

Furthermore I want to introduce the students to what pure interaction design is and what it takes to do it professionally. I chose our design project such that no one could simply fudge their way through using visual design or information architecture skills. How the interaction flows in time is our focus.

The title, interaction design for the real world, reflected that once again the course was all about working on a real design project on real software. New this year was the increased ‘real world’ focus, where for many aspects I related to the students how these are ‘business as usual’ for interaction architects: to deal with certainty with the uncertainties of any given project.

time + space

The time set‑up was different this year, with three full days of working together at the FH and the students sending me their final work two weeks after that. Also different this year was that my course was fully booked, with sixteen students participating.

I divided the students into four design teams and used our three days together for intense and full‑time design work (hey, just like the real world…), using the structure: day 1: from 0–to–100 on the project, ending in an expert evaluation; day 2: brainstorming to create as many solution options as possible; day 3: narrow down to a single design concept.

Each team had to then deliver a design concept document two weeks later. Looking at what could be achieved in three days, I set the scope of this document to be a project‐internal briefing, for the principal interaction architect.

GIMP + vector

On to our design project. I want that the work produced during this course ends up contributing to the real word. So I chose GIMP as our ‘client’ to design for, because I can ensure there that interaction design is advanced towards implementation.

During LGM the GIMP team discussed readying last year’s SoC vector layers code for inclusion in GIMP v2.8. That was exactly the type and size of interaction design project I was looking for. I will now introduce the material my students started out with.

The proof‐of‐concept code allowed to take a path:

and convert it to a vector layer, receiving an outline stroke and/or a fill:

the shape of the vector can be changed for ever like any path, and when not edited display like this:

the fill and stroke parameters could be found and changed in one dialog:

and that was it.

spoiled for choice

The exact set of functionality (aka features) has a major impact on the interaction design and interaction architects have to rightsize this set before even starting to think about interaction. Usually this means fighting too many features, since anybody—developers; users; marketing folks; managers—can come up with more features.

In our design project, there was plainly not enough functionality to be useful/usable. So before the interaction designing started, each team sat down to brainstorm and then decide on their essential set of functionality. What struck me at the start of this phase was to hear talk of ‘window’, ‘click’, ‘button’, ‘slider’, et cetera.

That had to stop immediately. It took a couple of rounds of explanation—‘it is just a boring list of what functions it has’—to get the cold, analytical side out of my students. But then they were on top of the functionality, as it should be. As we will see next, functionality like vector layer management, managing and combining shapes and working with simple shapes (rectangle, ellipse, etc.) was integrated.

four team results

Now the grand finale, the results of the four teams. First let me make clear that the work of all four teams is available under GNU Free Documentation License 1.2. I present them in team order.

team one

• members: Andrea Chavez, Christoph Bischof, Julian Tomaselli + Tamara Christina Blumer
• design concept document (pdf, 2.7 MB, limited pdf viewer compatibility)

Team one was the one with the hottest debates. That may sound a bit inharmonious, but it was good to see the whole team so worked up about making good interaction.

The main theme every team fundamentally had to deal with was the integration of the new vector functionality with the existing interaction, like that for paths and geometry transformation. This team solved it by introducing a shape tool in the toolbox and using the path tool for advanced vector manipulation.

Team one was the fist of several teams to arrive at the solution to use a library dialog for selection and management of a large set of user‐defined vector shapes. This works analogue to how brushes are selected and managed.

Apart from a doing a good job overall of keeping all the aspects of the interaction together, I especially liked their use of the modern GIMP sizing handles for the simple shapes.

team two

• members: Sara Geller, Laura Ramirez, Jakob Tripolt + Florian Wenger
• design concept document (pdf, 9 MB)

Team two shows us that all it takes to deliver a detailed design concept is pencil and paper. Funny enough this team, that did the most note‐taking, also resisted for the longest time to use the pinboards that I strongly recommended for gathering the results of the brainstorm phase. At the end, they did.

Team two integrated shapes and vectors into the path tool, giving the path tool different modes.

Overall they went deepest into a lot of interaction issues and solutions. I really liked their serious workflow analysis.

team three

• members: Mona Seiffert, Giselle Anahi Salinas, Markus Angerlehner + Anna Weiss
• design concept document (pdf, 732 KB)

Team three shows that it does not take a long document to get the core concept across. To my surprise this team dived into making pixel‐perfect mock‑ups of their ideas quite early in our design process. That was another thing that had to stop immediately.

In our high‐pressure environment where the teams had just enough time to arrive to a solid concept, the focus has to be on staying agile and creative. While all the time the scope of the design gets wider and the big picture becomes clearer, any part‐solution needs to be hot‐swappable for a better one until the very end. You can do that when solutions are pencil sketched, not so easy when they are visually designed with considerable effort.

‘Keep it brief’ is what I asked for the design concept document. But vector layers turned out to be a design project that was more multifaceted than the humble introduction would let one believe. So apart from the core concept, a lot of peripheral interaction issues needed addressing, as the other teams did.

team four

• members: Daniel Corn, Katherina Donner, Marko Heijnen + Lena Seeberger
• design concept document (pdf, 528 KB)

Team four introduced not one, but two new tools in their design concept. And for good reasons, like addressing smooth workflow, or an appreciation of the elegance of the free‑poly selection tool. So they introduced a freehand tool besides a shape tool, the former converts freely drawn shapes to stroked/filled vector shapes.

Also this team did a a good job overall of keeping all the aspects of the interaction together and I am quite taken in by their clean and effective little storyboards.

after the project is before the project

From the moment I selected vector layers as our design project down to the moment I am writing this, I have kept out of actively designing this feature, by simply not starting to. This in order to stay as objective as possible, while working with the students; while grading their work; even while describing it here.

But now it is time for me to get started. I will take the concepts of all four teams as the starting point for my interaction design of vector layers for GIMP. I’ll keep you up to date on that here.

Finally, I would like to thank the students for their hard work during the three intense days that we spent together. It was great to feel the vibe and I had a great time.

Labels: architects, fundamental, GIMP, practical, teaching

—ps

3 comments, link‐ins, forward this posting: