Current Issue
Culture, Internationalisation and Usability - II - 2 March 2008
by SESUN Team
Read this article in Chinese (translated by Sean Liu)
Authors:
Zhengjie Liu, Andy Smith, Kerstin Röse, et al.
3.2 Germany
The focus group conducted in Germany consisted of usability professionals with various backgrounds, e.g. from the automobile industry, web-design, the mobile industry or from usability consultancies. Participants invested their free-time and were not financially rewarded in any terms.
Even though most participants were engaged in the field of usability engineering for many years already, they were lacking explicit experience in cross-cultural usability engineering. This strongly affected results obtained by the sessions.
While within the first session on UE rich and detailed information regarding current and best practices was gathered and deeply discussed, the XUE session was a rather shallow and shortened discussion on the surface with less insightful results.
3.2.1 Usability Engineering Process
The participants jointly developed UE process was visually finished-up by the researcher with consistent feedback of the participants after the focus group session. The overall process consists of 8 stages with three stages being critical iterative evaluation stages and three critical decision stages.
The first stage is the product definition or idea, which initiates the whole process. Consequently this stage is to be seen as the first milestone within the development process. The product idea can be based on someone’s inspiring spark, but usually is derived from carefully observing the market situation. Within this stage the target market, the target users and the target price to be developed for are defined. Also the products unique selling propositions (UPS) are to be fixed. Major stakeholders of this stage are marketing and management.
Within the next stage of requirements analysis all data about the user and the context of use are gathered. This data represents the foundation for the further product development and also serves as decision basis for upcoming evaluations. Major stakeholders here are usability analysts.
Based on the prior stage, first design propositions are developed as conceptual drafts. It is to point out that several drafts are developed simultaneously. The conceptual draft consists of basic functions of the product, interaction modes and applied metaphors. Designers and usability analysts are the major players within this stage.
The developed drafts then are evaluated. Consequently this stage is called draft evaluation, which aims at the identification of the best drafts in regard to collected requirements in stage two. Beside the collected user-requirements the expected usability-quality of the drafts are major decision criterions. Usability-tests, expert-evaluations, simple prototypes (e.g. paper-prototype) and scenarios are applicable methods here – all this with real user participation, where possible. But also technological constraints, free production capacities, technology and material availability play an important role when evaluating the drafts.
The second major decision of the development process, which is consequently the second milestone of the latter, is the choice of drafts. Based on the conducted evaluation in stage four the best drafts are selected for further development and are hereinafter referred to as favourites. For practical reasons and due to limited resources the amount of favourites usually is less than four.
For the selected favourites detailed implementation requirements are defined within stage number 6. These implementation requirements should include all requirements necessary for the realization of the draft. That is producibility, supplier scheduling, laws, norms and other official regulations, etc.
Based on these detailed requirements description for each favourite these are evaluated once more within stage seven - favourite evaluation. Here the selected drafts are further scrutinized. In the following decision stage it is decided which favourite will later be realized.
The detailed description how this is to be done is developed in stage nine where the product specification is to be defined. The specification represents the detailed description of user-requirements as well as the requirements of the product manufacturer. In some cases one might just realize within this stage that the product is impossible to develop as defined within the draft. Even though this risk should be minimized through prior evaluations, in this case the draft development needs to be revisited and scrutinized to resolve identified obstacles.
Based on the specification more sophisticated prototypes are developed in stage ten - prototype development. For user-centred product development prototypes are of utmost importance as they facilitate the application of sophisticated usability tests and allow the practical engagement of the user as opposed to the rather theoretical user-engagement within prior stages. Furthermore prototypes are highly required by designers and manufacturing specialists in order to test different possibilities of realization.
Consequently, the stages of prototype development and evaluation represent a highly interwoven and iterated sub-process, which might also affect the product specification. Once a certain aspiration level that satisfies all stakeholders is reached, the specification is released. For quality control within the development process, the product specification is of utmost importance as every unit involved in the follow-up development stages will be able to deliver the specified function with the defined quality within the specified time. Naturally, this must be considered as an important milestone within the UE-process.
On this basis, the product implementation begins. Here all the requirements to launch the product are to be made. This includes for example the development of production-tools, the implementation of distribution channels, the development of marketing campaigns etc.
Once all this is implemented the product is ready to go into production and can be released.
Regarding applied methods, usability-professionals in Germany differentiated between methods that are effective to apply and methods that are ‘nice’ to apply, that is methods that are fancy, popular or are fun to apply. Interestingly, most professionals seem to perceive effectiveness and niceness as mutually exclusive as the table below shows. Instead of merely accepting this as a fact, this finding should rather encourage usability professionals and researchers to develop more methods in which niceness and effectiveness reinforce each other.
Table 3: Favourite Methods (less points means higher rating)
The most effective methods, as perceived by the participants, were expert evaluation and context analysis followed by scenarios and eye tracking. From nicest to less nice methods participants ranked scenarios higher than eye tracking and context analysis which seemed nicer to them than expert evaluations.
3.2.2 Cross-Cultural Usability Engineering Process
Regarding the cross-cultural usability engineering process participants were somewhat troubled to agree upon one state of the art definition. One reason for this probably can be seen in the focus group composition and current best practice in this field. Most multinational companies developing for international markets, and most participants were employed by multinationals, utilize facilitators within the target market for analysing requirements of the user, the context and the task. Hence, participants had little to no experience in the early stages of the development process.
They pointed out, however, that except the stage of requirements analysis other stages of the development process differed only marginally from the UE-process as described above. The difference in analysing requirements for cross-cultural usability engineering thus is that this is done by local usability experts within the target-market. This implies also that in the follow-up stages the user is virtually not involved.
3.3 United Kingdome
The focus group conducted in the UK was made of usability professionals with experience in different domains such as e-commerce, mobile devices, domestic appliances and e-government, among others. Participants accepted to contribute to this project with their consultancy time and were not financially rewarded.
All participants except from one had experience of cross-cultural usability engineering and were able to give their views and compare key differences with their standard of UE.
The session was similar to that conducted in Germany: rich and detailed information about UE current and best practices was gathered and discussed; then a shorter session followed in which discussion only revolved around the key differences between XUE and UE.
3.3.1 Usability Engineering Process
The participants jointly developed an ideal UE process coordinated by the researcher with feedback from the participants during the focus group session. While the process presented is not necessarily applied by most UE professionals, it reflects in the participants’ experience of what constitutes best practice. The overall process consists of 6 stages with three stages having to go through iterations before moving to the next level (Figure 3).
Figure 3: Usability Engineering process in the UK
The first stage is the product vision or idea which initiates the whole process. It usually comes after marketing research discovers an opportunity for a product or solution. This idea is sponsored and driven by a ‘vision owner’, who will follow its evolution through the entire product development lifecycle. Roles commonly involved at this stage include the client and marketing staff. The latter refine this idea with methods such as focus groups, surveys and competitor analysis.
The stage of assumptions validation consists of the production of a validation document in which the needs and user requirements derived from the product vision are validated, and which in turn generates new requirements and cancels others. This implies the gathering of requirements through direct contact with users in the form of ethnographically inspired methods, direct observation and discussion with users and members of the project team of user models, scenarios and personas. User researchers and software engineers are key roles at this stage. The latter are of special importance as they establish the technical constraints and objectives of the product being developed.
Based on the prior stage the functional scope document and interaction design requirements are produced. The production of these documents constitutes the stage functional scope definition. User researchers and software engineers iterate various versions of this document before a more definitive version is proposed for the development of prototypes. They conduct workshops with members of the project team and risk analysis to evaluate the feasibility and accuracy of this document in terms of the original product vision and end-user requirements.
It is worth noting that the functional scope document is not necessarily a single entity, but it could be represented by several documents covering different aspects of the product, including more specific interaction design requirements. The important point is that this document or set of documents mark the start of the prototyping stage.
Based on specifications from the scope document, prototyping begins. This can be considered to be a single or multiple prototypes exercise, but the key point is that their evolution is driven by user feedback. Typically, design and usability roles are the main players at this stage with an important input from the vision owner. The prototypes are evolved and evaluated with a variety of methods: the ones mentioned by UE professionals were in situ observation, card sorting, user testing, interviews and story telling. The products of this stage can be wireframes, site maps, information architecture, low and high fidelity prototype; each will be the result of a particular iteration cycle. A user-tested high fidelity prototype usually marks the start of the development stage. However, it is hard to indicate the specific point at which this latter stage begins as it usually overlaps with the later stages of prototyping.
The development of the product is mainly done by software engineers and graphical interface designers. Bringing in users at this stage is usually difficult but quick usability tests are often needed to for specific modules or aspects of the product. This is done by expert reviews and walkthroughs, rapid testing with user proxies or representatives or other ‘guerrilla’ methods. This last set of methods can consist in bringing very few users in for a quick review or role playing games among designers and developers.
Near the end of the production development cycle formal user testing is done to validate requirements and identify possible areas of change before release. However, very little change can be done at this stage and any important requirement that cannot be implement are postponed for a new version of the product in certain cases.
The post-deployment stage involves a broader set of roles. Once the product is released to end-users, its performance can be measured in terms of clear usability metrics. The use of web analytic tools is very popular to measure the behaviour of users in websites. Other automated data logging tools can be used to gather information on longitudinal user behaviour, including the use of mobile devices. These data can be used to generate performance reports and other type of documents that include qualitative and quantitative user feedback. User researchers and marketing roles are normally in charge of designing data gathering tools at this stage. Customer care and helpdesk roles also play an important role as they are in direct contact with customers and end-users. Business analysts can identify new needs after deployment and make the case for a new version or a new product. These findings trigger the beginning of a new UE cycle.
The most popular and widely used methods ranked by UK participants are presented in Table 4. The method with higher ranking is field studies. This indicates the prime value given to direct feedback from users in their own natural context. Most UE professionals converged on the situated nature of UE requirements. Interestingly, the second most valued method by UE professionals was ‘client viewing’, which refers to the client and head developers observing how ‘bad’ their product is. They see this as an important political and pragmatic step in the process gaining support from vision owners and decision makers to establish a more user-centred approach in the development lifecycle.
In this case, there was no distinction between ‘nice’ and effective methods. All the participants put forward those techniques they considered to be useful in the sense of leading to a user-centred product.
It is worth noting that the process and methods emerged from this session were presented by UE professionals in close relation to the different aspects of product development, marketing and client constraints, not just usability.
Method | Points |
Field Studies | 5 |
Client viewing | 7 |
Lab studies | 10 |
Task analysis | 12 |
Wireframes and Prototyping | 14 |
Pilot Tests | 19 |
Cognitive Walkthrough | 21 |
Table 4: Favourite Methods for UE process (less points mean higher rating)
Comments made
Possible Related Articles:
Latest Comments