Exploring product design quality control and assurance under both traditional and crowdsourcing-based design environments

Abstract

Small and medium-sized enterprises face the challenges that they do not have enough employees and related resources to produce high-quality products with limited budget and time. The emergence of crowdsourcing provides an opportunity for them to improve their products by leveraging the wisdom of a large community of crowds, including their potential customers. With this new opportunity, product design could be conducted partially in a traditional design environment (in-house design) and partially in a crowdsourcing environment. This article focuses on product design stages to investigate what key factors affect product design quality and how it can be controlled and assured. First, we define the concept of product design quality and then identify its attributes and sub-attributes. Second, we separately survey key factors affecting product design quality in traditional and crowdsourcing-based design environments, quality control approaches/theories and quality assurance policies in traditional design environment. Third, a comparison of product design quality issues between the traditional and crowdsourcing-based design environments is progressed focusing on various aspects influencing product design activity quality. Finally, we discuss product design quality control approaches and quality assurance policies, quality control challenges and corresponding solutions in crowdsourcing-based design environment.

Keywords

Product design quality control assurance crowdsourcing collaborative challenges

Introduction

Product design is an essential activity in the modern life as it can generate and develop ideas through a process that leads to new products to meet the ever-changing user needs and expectation.^1,2 Products with high quality will not only meet needs of both manufacturing enterprises and various users along the product life cycle but also bring good user experience and better social and environmental benefits to our society, thus helping the enterprises remain globally competitive in the fierce competition. Therefore, the control of product quality through the whole life cycle is a vital activity to enterprises. The aim of product quality control is to make a compromise between cost and product quality. However, product quality is a nebulous concept and it may be viewed in different perspectives.³ And the existing literature usually pays too much attention to quality control of manufacturing and activities after the product is manufactured. For example, anastasia³ shows that quality control of product quality is controlling production, carrying out repairs and warranty costs through defect discovery and maintenance, which ignores the importance of product design quality. From the incline of product quality through the whole life cycle (see Figure 1) in literature,^4,5 it is clear that product design quality is the key factor determining product quality.

As a stand-alone product is not sufficient to fulfil customer requirements, product service systems (PS2 or PSS) are usually combined with the product to provide product-related services such as maintainability, repair, update and quality warranty throughout the whole life cycle to better fulfil user requirements.^6,7 Therefore, we define product quality as the sum of product design quality, manufacturing quality, maintainability and recycling. Correspondingly, we consider that the product life cycle consists of four stages, that is, design, manufacturing, maintenance and recycling. The relationship among these terms is shown in Figure 2.

In manufacturing aspect, many studies have been conducted to ensure the product quality from various perspectives such as manufacturing methods and modelling approaches,⁸ the influence of human factors,⁹ key technologies of intelligent design for customized products¹⁰ and challenges and future of manufacturing in engineering.¹¹ Key factors affecting manufacturing quality can be classified into two categories: hard and soft factors.¹² After the product is manufactured and sold out, after-sale service, such as maintenance and recycling, plays a critical role in the successful marketing of many products as it enables customers to get the full value from its products.¹³

After-sale services attempt to resolve problems met by a customer, for example, product failure restoration and problem with using the product, which will cause dissatisfaction if the problem is not well resolved.¹⁴ Due to the increasing importance of after-sale service, Rolstadaas et al.¹⁵ discussed various aspects of after-sale services with regard to business model, service-delivering methodology, performance metrics, service portfolio and product planning and control. While Takeuchi and Quelch¹⁶ summarized the measures that are used for ensuring the quality of after-sale services, Markeset et al.¹⁴ considered that the key factors affecting maintainability include cost consideration, technological consideration, human factors, statutory requirements and accidents and so on. Product collection method, local authority facilities, charging structure and support, geographical location¹⁷ are considered to influence recycling quality. Compared to manufacturing, at maintenance and recycling stages, there are relatively less studies focusing on how to integrate product design quality into product after-service qualities despite that design for manufacturing, design for assembly and design for disassembly are proposed as design guidelines.

In conventional design environments, product design activities are usually performed by an in-house design team consisting of individuals with different expertise and experience. This kind of design collaboration mode is effective for large-scale enterprises but not for small and medium-sized enterprises (SMEs) who are lack of enough skilled employees and related resources to support their product design activities. Benefitted from the advantages of crowdsourcing, such as cost-effectiveness and global participation of crowds, SMEs show great interests in leveraging crowdsourcing-based platforms to perform their product design activities or improving their products as crowdsourcing has the potential to overcome their shortcomings in employees and related resources. In parallel to traditional product design processes, more and more SMEs^18–20 are moving part of their product design processes on to a crowdsourcing platform, benefitting from the participation of a large number of crowds including their potential customers to the fast speed and cost-effectiveness of the solutions generated. Although crowdsourcing has shown great potential to create more values in product design domain, there are no crowdsourcing platforms that fully support product design activities.^18,21 Thereby, SMEs just rely on these platforms to perform some specific product design activities such as idea generation and selection. Until now, many researchers have devoted to exploring how to support product design processes in crowdsourcing context.¹⁸,^20–23 Since product design research over a crowdsourcing platform is still in its early stages, there are few studies on how to control the product design quality over a crowdsourcing-based platform.

This article intends to give a definition of product design quality, survey the key factors affecting it in both traditional design environment and the crowdsourcing-based design environment and conduct a comparison between them. It also identifies product design quality control challenges over a crowdsourcing platform and investigates possible solutions to deal with these challenges.

Our main contributions in this article are fourfold:

Giving a hierarchy of product design quality attributes and defining their sub-attributes;

Surveying the key factors affecting product design quality, quality control models/approaches and quality assurance policies in conventional design environments, and summarizing product design quality control studies in terms of four research focuses: the management of information, the sharing of information, quality control approaches and quality assurance policies;

Analysing key factors affecting product design quality in crowdsourcing context and comparing the traditional design environment and the crowdsourcing context in terms of various aspects affecting product design quality;

Discussing product design quality control challenges over a crowdsourcing platform and proposing corresponding solutions to deal with these challenges.

The rest of this article is organized as follows. The ‘Definition of product design quality’ section gives a definition of product design quality, in which key quality attributes and sub-attributes of product design quality are given. The ‘Research method’ section presents our literature searching rules, the analysis results of found literature and the classification results of some literature that are conducted to ensure quality of different design quality attributes. The main contributions of this article are described in the ‘Research findings of quality control studies in traditional design environments’ and ‘Research findings and quality control challenges in crowdsourcing context’ sections. The ‘Research findings of quality control studies in traditional design environments’ section presents research finding in traditional design environments, including key factors affecting product design quality at different level of product design quality hierarchy, key quality control theories/methods and quality assurance policies. The ‘Research findings and quality control challenges in crowdsourcing context’ section mainly describes the research finding in the context of crowdsourcing, including key dimensions and factors affecting product design quality, and discusses quality control and assurance policies in crowdsourcing context and the product design quality control challenges in crowdsourcing context. Final section concludes the article and indicates the benefits of crowdsourcing in product design and development (PDD).

Definition of product design quality

At the product design stages, quality control is one of the most important activities as it ensures to achieve the design goal.⁴ The aim of product design quality control is to check the key aspects of design quality against a set of standards or specifications. The quality control process can be achieved by a three-step iterative loop of design process in its simplest terms:²⁴ ideate, prototyping and evaluate phase, which is shown in Figure 3. In this process, design idea/concept is first generated, then it is prototyped, after that the design team would evaluate the design via prototypes. Finally, the design feedback generated from evaluation via prototypes will be fed back to corresponding designers for design improvement. The iteration of the design process helps designers discover unknown variables and their interrelationships,²⁵ and this iteration process is helped by idea prototypes with different fidelities ranging from two-dimensional (2D) sketches, three-dimensional (3D) computer aided design (CAD) models, 2D/3D mock-ups and printed 3D models. Lim et al.²⁶ have explored the role and characteristics of prototypes in the domain of design. And impact of prototyping on design results has been investigated by Dow²⁵ and he found that prototyping can initiate a conversation with the space of design possibilities. By integrating design feedback generated from prototypes that represent a class of a generalized heterogeneous grouping of elements derived from alike design cases that provides the basis for the start and continuation of a design into their designs,²⁷ the designers can improve their current design concept. However, since the interpretation of product design quality varies from person to person, it is hard to give it a specific definition. Nevertheless, many studies have tried to define it. The various definitions of product design quality can be found in Table 1.

Table 1.

The definition of product design quality.

Source	Definition or arguments of product design quality
Zhu et al.⁵	Design specification should conform to the requirement of customers
Zhu et al.;²⁸ BBC	Design requirements reflect the voice of the customer or the demands of the market
Aas;²⁹ Mrugalska and Tytyk³⁰	Design object satisfies its specification
Salimun et al.;³¹ ISO 9000: 2005	The degree to which a set of inherent characteristics fulfils requirements
Spacey³²	The value of a design to customers
BusinessDictionary¹	Level of effectiveness of the design function in determining a product’s operational requirements (and their incorporation into design requirements) that can be converted into a finished product in a production process
IBM; Hermans and Liu³³	The degree in which customer requirements are met

BBC: british broadcasting corporation; ISO: international organization for standardization; IBM: international business machines corporation.

According to the relationship between Maslow’s hierarchy of needs and emotion design,^34,35 Bradley³⁶ proposed a five-level design hierarchy of needs, including functionality, reliability, usability, proficiency and creativity. Most of the existing product design quality control approaches are used to ensure the lower levels of user needs, that is, functionality, reliability and usability. Nevertheless, in the domain of industrial design, the maintenance of product is an important aspect affecting quality as well. Therefore, the maintainability should be taken into consideration when designing the product and it is treated as a key quality attribute in our product design quality hierarchy. In addition, we treat proficiency as a sub-attribute of functionality. Figure 4 shows the product design quality hierarchy of this research.

Taking both Table 1 and Figure 4 into consideration, this article defines product design quality as follows: the degree in which a set of inherent design characteristics of product meet requirements of hierarchical design quality of functionality, reliability, usability, maintainability and creativity; commercial requirements; and the required standards. In this definition, we consider not only the requirements of target audience (users) but also the requirements of other role players such as workers involved in the manufacturing, maintenance and recycling stages. From the perspective of product whole life cycle, all involved workers in the product life cycle are customers of the product.

In Figure 4, the highest level of design quality, creativity, is the last but very important aim of product design. Philips³⁵ and Schütte³⁷ have showed that making customers feel satisfied and happy could bring more benefits both in monetary and emotional aspects. Although a lot of research has been conducted on the controlling of one or two combinations of these five quality attributes, the ultimate goal of product design is to ensure the product quality in these five aspects. The five levels of design quality hierarchy are treated as design quality attributes in this research. Each design quality attribute consists of several sub-attributes. Referring to product quality characteristics defined by ISO 25010 Standards³⁸ in Software Engineering, the common types of design quality in Table 2 indicated by Spacey³² can be treated as sub-attributes of these five design quality hierarchies. The aim of quality control at product design stages is to ensure the achievement of these five aspects. There are already related studies focusing on ensuring single and specific design quality attribute (see Table 2). In reliability design, there is already a review paper about analysing the research progress based on axiomatic design theory.³⁹

Table 2.

Design quality attributes and sub-attributes.

Design quality attributes	Quality sub-attributes	Definition
Functionality^4,30,40,41	Functional completeness	The extent to which the functions cover all specified tasks and objectives
	Functional correctness	The extent to which the product provides correct results with the needed degree of precision
	Functional appropriateness	The extent to which the functions facilitate the accomplishment of specified tasks and objectives
	Time behaviour	The extent to which the response and processing times meet requirements
	Resource utilization	The extent to which the amounts and types of resources used by a product meet requirements
	Capacity	The extent to which the maximum limits of a product meet requirements
	Sustainability/environment friendly	It is a requirement to the designer. The product must be friendly to environment
	Safety and security	The extent to which a product is safe for its users
Reliability^{30,39,42–44}	Maturity	The extent to which a product meets needs for reliability under normal operation
	Fault tolerance	The extent to which a product continues in a reasonable way when errors occur
	Availability	The extent to which a product is operational and accessible when required for use
Usability^45–49	Appropriateness recognizability	The extent to which users can recognize whether a product is appropriate for their needs
	Learnability	The extent to which a product can be used by specified users after learning
	Operability	The extent to which a product has attributes that make it easy to operate and control
	User interface aesthetics	The extent to which a user interface enables pleasing and satisfying interaction for the user
	Accessibility	The extent to which a product is useful for everyone in the context of use
Maintainability^50,51	Analysability	The extent to which the product behaviours and performances are diagnosable and predictable
	Testability	The extent to which a product facilitates the establishment of acceptance criteria and supports evaluation of its performance
	Modularity	The extent to which a product is composed of discrete components
	Reusability	The extent to which a product or its components can be reused after disposal
	Modifiability	The extent to which a product can be modified without introducing defects or degrading existing product quality
Creativity^52–54	User interaction/experience	Intangible elements of quality
Creativity^52–54	Emotional durability	A design that people value at an emotional level such that they do not easily throw it out

Since reliability, usability, maintainability and creativity must be relied to functionality, more attention is often paid to the creation of functions and their combinations that satisfy established needs rather than the other four design quality aspects at earlier product design stages.⁴² In order to obtain product designs with high quality, the design process has always been controlled. However, what factors affect design quality at different product design quality levels and what aspects should be controlled are still not clear. Therefore, the following method is adopted to extract research focuses from the existing literatures on quality control of product design at product design stages in both traditional design environment and crowdsourcing context.

Research method

All reviewed literature is searched from the following databases: Web of Science, ScienceDirect, ACM Digital Library and IEEE Xplore Digital Library. The keywords used to search articles are organized in three descriptor groups with rules listed below. In this review, Boolean operators ‘AND’ and ‘OR’ are adopted to make logical searches. $R_{1}$ , $R_{2}$ and $R_{3}$ use the rule of ‘OR’ to represent the three descriptor groups, respectively. $R_{0}$ is the sum of $R_{1}$ and $R_{2}$ or $R_{1}$ and $R_{2}$ and $R_{3}$ with the rule ‘AND’. For example, several keywords combinations such as ‘industrial product design’ with ‘quality control’, and ‘conceptual design’ with ‘design for functionality’ and ‘crowdsourcing’ have been examined

\begin{matrix} R_{1} = keywords \in (product design OR conceptual design \\ OR detail design OR idea development OR idea generation \\ OR process design OR idea prototyping OR idea evaluation) \\ R_{2} = keywords \in (quality control OR design for \\ functionality OR design for usability OR design \\ for reliability OR design for maintainability \\ OR design for creativity) \\ R_{3} = keywords \in (crowdsourcing OR cloud OR cloud - based) \\ R_{0} = keywords \in ((R_{1} AND R_{2}) OR (R_{1} AND R_{2} AND R_{3})) \end{matrix}

As a final retrieval result, a total number of 125 literatures from six categories of resources, that is, journal articles, conference proceedings, book sections, related webpages, standard and thesis, are found and classified. With the literature searching rules, the found studies could be classified into two categories according to the context: the traditional design environment and the crowdsourcing context. After getting the data set, all found literatures are screened manually to select out studies that are conducted at product design stages. As a result, 118 literatures from journal articles and conference proceedings are selected as most closely related to our research objective. The number of studies in these two different contexts are 100 and 18, respectively. Since we have reviewed key crowdsourcing technologies for PDD regarding to organization structure, solution evaluation, workflow management and quality control in literatures,^21,55 here only 18 literatures (10 journal papers, 7 conference papers and 1 book section) are mentioned to analyse the factors affecting product design quality in crowdsourcing context. Table 3 shows literature distribution of studies in the traditional design environment.

Table 3.

Literature distribution of studies in the traditional design environment.

Reference type	The number of references
Journal articles	63
Conference proceedings	16
Book sections	5
Related webpages	14
Thesis	1
Standard	1
Total	100

For these research papers appearing in journals and conference proceedings at product design stages in traditional environments, the TagCrowd was applied to extract keywords distributions to find out the research focuses. During this process, the searching keywords were excluded. The rest 258 words or word groups were collected and analysed. Figure 5 illustrates the top 50 words with the highest frequency. The result shows that the top-ranking keywords are management, collaborative, process, information, model, data, sharing, communication, knowledge, life cycle, systems and conflict. Based on this result, the extracted key research focuses in controlling product design quality in traditional design environments include the management and communication of relevant information and quality control approaches. After reading through these studies, another research focus, quality assurance policies, is found. The detailed analysis against each term is presented in the following sections.

The keywords analysis result from the 18 literatures related to crowdsourcing context is shown in Figure 6. The top-ranking keywords are communication, collaborative, co-creation, incentive, participation, motivation, feedback and assessment. It is clear that the research focuses in the crowdsourcing context are crowd participation, incentive mechanism, communication, feedback and assessment. However, the studies about these research focuses have been analysed by Niu et al., here we no longer describe them. Please refer to Niu and colleagues^21,55 for more details about these research focuses.

Research findings of quality control studies in traditional design environments

Traditionally, a successful product is achieved by the effort of a team.⁵⁶ A product design task is performed by a well-designated team, which consists of employees from different functional departments such as marketing, finance and technical department.⁵⁷ The budget for design, the capability of designers and the tool designers used definitely influence the product design quality at every design stage. Except these, there are many other factors⁵⁶ that are outside the direct control of designers affecting at least one attribute of product design quality, for example, product development strategies, market orientation, technology and top management support. For a product design team, the support of design process, team environment and other assistance is to support team members to access the required knowledge and to utilize the knowledge for product design.⁵⁸ All these factors⁵⁶ have an important role to product design quality. They could be classified into three key aspects in terms of design process, team management and information management. The factors affecting product design quality in terms of these three aspects (design process,^59–65 team management,^65–67 information management^66,68,69) are shown in Figure 7.

Knowing key factors affecting design quality in design process, team management and information management, we further investigate factors affecting product design quality at different quality levels, key quality control theories/principles/methods/tools and key quality assurance policies, which are shown in Figure 8. Some factors^70–75 affect single and specific design quality attribute and the others influence one or more attributes of product design quality.⁴ In order to better understand how to ensure the product design quality, this article takes into consideration control theories,³⁹ approaches,^{41,43,44,47,48,52} principles⁴ and tools.⁷⁶ Quality assurance is a set of policies through product life cycle to make sure the product can systematically meet the quality standards and thus leave an impression to the customers that companies provide high-quality products that are reliable, durable and dependable.¹⁴ The quality assurance policies on both supply chain⁷⁷ and after-sale service^14,78 have an influence on final product quality. It enables the quality of products according to Plan–Do–Check–Act ethos.⁷⁹ Case studies from Dale and Duncalf⁸⁰ found that nearly all companies have quality assurance policies that usually exist in the form of written documents or verbal communications.

In order to ensure the quality of these five design quality attributes, the design process needs to be well controlled. Therefore, Tag Cloud (a data visualization tool) is utilized to analyse keywords from related literatures and four research focuses are identified. Figure 9 shows the classification of related literatures according to the research focuses: information management, information sharing, quality control approaches and quality assurance policies. The sections ‘Information management’, ‘Information sharing’, ‘Quality control approaches’ and ‘Quality assurance policies’ present the related studies under the terms of these four research focuses.

Information management

Information management is to identify, capture, evaluate, retrieve, maintain and share all of the information assets in an enterprise.⁸¹ The effective management of information can help the enterprise find out valuable information, thus earning more economic benefits. Whatever scale the enterprises are, they should pay attention to the management of relevant information. For SMEs, they usually collaborate with other companies that are geographically distributed; thus, effective information management is needed to achieve a specific market objective.⁸² For large companies, they need to well manage the information to support the collaboration of employees.

With the objective of knowing what kind of information is interchanged during a collaborative design project, Giannini et al.⁸² analysed the design activities currently carried out in the technical offices of typical SMEs and found that designers always needed to communicate a subset of the whole information describing the product to be designed to co-designers. In this process, it is necessary to effectively and securely track, control, manage and share the rich information.⁵¹ The management of information at design stages is vital to the success of final product and it is often achieved by PLM (product life cycle management) systems. PLM systems are gaining acceptance for managing all information about products acting as a collaboration result of designers from different departments or enterprises throughout their life cycle. The existing studies have proposed some information modelling framework trying to identify design information and product knowledge. The proposed information frameworks/models and the information managed are shown in Table 4.

Table 4.

The existing information framework/model.

Framework/model	Information managed at design stages	References
Based on CPM and its extensions, OAM, DAIM and PFEM	Product information, design rationale, assembly and tolerance information, the evolution of products and product families	Sudarsan et al.⁸³
Product manager	Product-specific knowledge, concepts pertaining to the description of the product whose design is in charge of the node and its lower level co-designers	Giannini et al.⁸²
Process manager	The description of the processes: needed activities, dependencies between them, rules to follow, time schedules and constraints, planning of work, participant organization and synchronization of all of them	Giannini et al.⁸²
Cloud-based design	Product data, customer feedback, market information	Wu et al.⁸⁴
Satisfaction importance evaluation model	Personal information, product feedback, feedback on service, feedback on product modules, extra comments	Mourtzis et al.⁸⁵
PDMM	Design data, material properties, geometric and topology models, dimension information, finite element analysis and optimization, process planning, scheduling	Zhang et al.⁸⁶
Quality function knowledge deployment model	Domain knowledge, design standards, design specifications, comprehensive knowledge, material, mechanical assembly, CAD technology and related design experience, the social background, technology development, production resources, schedule	Zhang et al.⁸⁷

PDMM: product data master model; CPM: core product model; OAM: open assembly model; DAIM: design-analysis integration model; PFEM: product family evolution model.

In this section, the following questions need to be considered for design quality control: the representation of information, the quality of gathered information and the management of conflict information.

The representation of information

During a product design process, there is a large amount of information available such as product information and knowledge (knowledge is extracted from the analysis results of information), process information, the version and status of design solutions and customer feedback. This information is often stored at designated servers and is organized through basic elements so that it can be easily handled and monitored. In order to better represent this information, Chandrasegaran et al.⁸⁸ defined it in three dimensions: formal versus tacit, product versus process and compiled versus dynamic (see Table 5). The aim of information representation is to communicate the attributes and benefits of the design concept and to help users judge whether the design concept matches his or her needs and interests.⁸⁹

Table 5.

The dimensions of information available during a product design process.⁸⁸

Dimension	Explanation
Formal	Embedded in product documents, repositories, product function and structure description, problem-solving routines, technical and management systems, computer algorithms, expert knowledge systems and so on
Tacit	Tied to experiences, intuition, unarticulated models or implicit rules of thumb
Product	Includes requirements, various kinds of relationships between parts and assemblies, geometry, functions, behaviour, various constraints associated with products and design rationale
Process	Design process knowledge refers to design methods in representing designs, providing mechanisms for realizing design details
Compiled	Gained from experience that can be compiled onto rules, plans or scripts, cases of previously solved problems and so on
Dynamic	Qualitative and quantitative knowledge: qualitative knowledge consists of commonsense reasoning, approximate theories, causal models of processes, general problem-solving knowledge and so on; quantitative knowledge consists of constitutive, compatibility, equilibrium equations, numerical techniques, closed-form equations and so on

The most common and classical representation of design concept is storyboard. Such a representation usually consists of a sequence of sketchy pictures with captions.⁹⁰ This representation is considered to be effective in communicating the role that the design would have in the lives of people. Another typical representation of design concepts is prototype. It helps minimize design errors that may otherwise occur both early and late in the process. Prototypes often help designers identify design issues and learn from failures, and they support both design concepts evaluation and design exploration.^26,91 Feedback elicited from prototypes often frames subsequent actions around the existing design solution.²⁵ However, new technologies, such as virtual reality, make it possible for designers to create more advanced representations through utilizing animations and videos, even early in the design process. These media integrate sounds, motion and light effects into the product design representation to enable an immersive feeling.⁸⁹

In user-centred design, the users should be taken into consideration as they can help to evaluate the design concepts holistically. Therefore, their feedback on design concepts is meaningful to corresponding designers. Buskermolen et al.⁸⁹ and Van den Hende et al.⁹² have explored the effect of different representations on user’s responses to early design concepts. Van den Hende et al.⁹² found that the presentation format of design concepts did not have significant effects on perception and comprehension of concept, but it has distinctive effects on absorbing the participants in the narrative world. After analysing the effects of visual quality and animation of concept representation on users’ responses to early design concepts, Buskermolen et al.⁸⁹ suggested that sketchy representations provided more elaborate feedback and suggestions grounded on past experience, while visually refined representations were more helpful in eliciting definite judgements.

The quality control of gathered information

During a product design process, high-quality product information is essential as low quality often makes product development delay and can negatively impact the overall quality of the final product.⁵¹ The gathered information may be in many forms such as document, dialogue, audio and video. In order to quantify the quality of a document, Culley et al.⁹³ have explored the existing tools for gathering and analysing information, and summarized information assessment criteria in a comprehensive list of 94 criteria. However, the gathered information in the product design process is not only presented in documents. The quality of information in other forms, such as web reviews and user feedback, needs to be measured as well. In addition, providing feedback to the corresponding designer is a key step in the product design process, and it promotes the improvement of product designs. Therefore, this section mainly concerns the control of feedback quality.

Although there are a lot of ways to gather feedback from customers, such as on-site observations, questionnaires and interviews,⁵² the proper measures need to be adopted to ensure the feedback quality as the quality of gathered feedback relies on many different elements, for instance, the gathering criteria and process, the reliability of participants involved in this process, information quality checks and so on.

Boess⁴¹ found that how people use products is quite different from the expectations of designers, so the users need to be involved into the design process. Their feedback is potentially valuable for designers. In addition, user review is an important part of information at product design stages. Both the designers and manufacturers should pay attention to these reviews as they contain information about product and service experiences and they have an influence on buyers making their purchase decisions.⁹⁴ It can be treated as feedback from users about use experience. For product designers, these potentially valuable reviews could help them to identify customer likes, dislikes and desires. Yagci and Das⁹⁵ introduce the design-level information quality measure to evaluate the content, complexity and relevancy of the product-related reviews. The number of reviews, sentences, words, noun words and feature matching noun words in a review database are found to be key determinants in measuring information quality.

The management of conflict information

When designing a product, designers need to take into consideration a lot of interdependent aspects such as functional requirements and geometrical, behavioural and structural features.⁹⁶ Each aspect of these has its own set of constraints in which conflicting or unsatisfied requirements may be contained, and it is hard for designers to oversee the various alternatives and constraints all the time. The final product design is a compromised result of such constraints. In collaborative design, due to the participation of many designers with different technical background and expertise and their interactions, they may have disagreements about proposed designs.^96,97 In addition, the product design data may change frequently until the final product is manufactured as more than one designer works on the same product project. The ever-changing data versions and status should be well managed. All these information are treated as conflict and the management of them is critical in collaborative design.⁹⁷

Barclay⁹⁸ argues that there were three common types of conflict identified in product design: (a) imagined or perceived conflict, (b) latent or substantive conflict and (c) affective conflict. All these three conflict types are accompanied by process-related conflicts. With better understanding of the conflict types and causes that lead to conflicts, the management of them can make the outcomes of conflicts constructive.

The conflict management process includes five phases: conflict detection, forming the conflict resolution team, negotiation management, solution generation and solution impact evaluation.^96,97 A lot of research about conflict management have been conducted in collaborative design, but most of them have proposed methods to support the conflict detection^99,100 and the conflict resolution.¹⁰¹ Quertani and colleagues^97,102 also developed solutions to manage the negotiation process by tracking product specification dependencies. In order to assess the impact of a selected solution on the product as well as on the design process organization, Quertani⁹⁶ proposed a process organization framework based on data dependencies network.

Information sharing

PDD always involves many participants with different professional knowledge and background such as marketing and engineering design. All these participants work together towards a specific design objective. Collaboration is found to be a key to enhance competitiveness¹⁰³ and to improve product quality.⁵ Communication and information sharing is the foundation for collaborative product design.⁸⁶ Information sharing disseminates information with a community, which plays a crucial role in information management in product design process.⁸¹ Effective information sharing drives organizational and individual learning, which in turn speeds up and improves the quality of product. Due to its importance to product design, here we present it in a separate section.

From the existing literature available, information sharing is found to be useful in helping individuals, teams and organizations to improve their work performance.¹⁰⁴ At the same time, effective information sharing can help short the time taken to introduce the products to the market, which will give the company an edge in the fierce competition.⁸¹ In order to help designers work together effectively, certain measures must be adopted to enable the exchange of design information. ISO 10303 has provided an ISO standard for the computer-interpretable representation and exchange of product manufacturing information. However, there is still no similar standard for the exchange of product information at product design stages.

During the collaborative design process, the design-related and process-related information can be classified into two categories: public and exchanged information.⁸² Public information describes and indicates the main characteristics of the product and can be treated as a set of technical requirements, to which all participants involved in the process can get access. While the exchanged information can only be seen by certain participants and their versions depend on the sender, for example, customer, co-designer or supplier, and the states of the project development. In information sharing, only exchanged information is considered.

Based on collected product and process information, proper information sharing and communication mechanisms enable designers and their partners to collaborate effectively, which is a critical determinant of collaboration.^83,105 The main contents of communications are the product data and the process data.⁸² In traditional collaborative design scenarios, the designers usually have to spend much time on communicating with their partners through emails or phones. As for the design-related data stored at designated servers, they have to be authorized to have access to them. In the work of Zhang et al.,⁸⁶ a data access mechanism from different perspectives, functional views, personal workspace, work table and personal storage space, is provided, and even from the same perspective, different users have different data access authorizations. However, the proposed product model can only support static product information, which cannot satisfy the needs of real-time collaborative design. As for the systems on which all information are public, the protection of intellectual property is threatened.¹⁰⁶ Mun et al.¹⁰⁶ suggest just share the essential data with their collaboration partners.

In collaborative environment, the sharing of information is usually achieved through various design representation formats such as 3D models, images, videos and XML extensible markup language (XML) files.¹⁰⁷ One problem that needs to be considered in this process is that the users should exchange their ideas as instantly as possible. Hasby and Roller¹⁰⁸ proposed a CAD system that can facilitate the conceptual-embodiment design stage in a collaborative manner. Through the system, the designers can communicate their opinions and ideas freely. In addition, the system must avoid the possibility of losing some information.⁸² In order to prevent this possibility, the system should automatically create a list of changes that the user has made and notify relevant users to update their information.

Quality control approaches

Quality control models/tools

Quality control is an activity throughout the whole product design and development process. The aim of quality control is to satisfy the customer needs as well as to decrease the design and development cost in terms of time and money.¹⁰⁹ In this process, customer requirements and product design specification (PDS) are benchmarks of the controlled quality of product design.⁷⁶ In order to control the quality of product design, many researchers linked customer requirements with quality characteristics. For example, Tang et al.⁷⁶ argued that quality characteristics are the key control factors in the whole product design and development process and the focus of quality control is on how to translate customer requirements into product quality characteristics. Similarly, Zhang et al.¹¹⁰ developed a vector-based mapping tool that can provide reasonable mapping among PDS, behaviour parameters and structure parameters. Chu et al.⁴ converted user requirements into the relevant technical requirements of design using quality function deployment (QFD) and house of quality (HOQ). Although customer requirements provide benchmarks for the designated product, one problem that cannot be neglected is that the customer requirements change over time. In order to understand how requirement changes propagate in the design of complex product systems thus helping to select best options to guide design, Zhang et al.¹⁰⁹ proposed a PDS-Behaviour-Structure-based design change model that can systematically analyse and search change propagation paths.

The quality control models can be classified into computer-oriented and human-oriented. Among computer-oriented quality control models, Tang et al.⁷⁶ translated customer requirements into product quality characteristics and achieved their quality control aim by controlling these quality characteristics. As for human-oriented quality control models, a typical one is Fujitsu’s conceptual model of Kansei quality.⁵² The model has six elements:⁵² product/service usage situations, stimuli produced in usage situation, somatic sensations (perception/cognition), personality and past experience, impressions/emotions, behaviour. The somatic sensations is the contact point between the user and the outside world. These elements are controlled in the Plan–Do–Check–Act cycle.

Product design optimization

During product design and development process, the initial design may be functional, but it may be far from optimal in terms of quality and cost.⁴⁴ Hence, it is necessary to optimize the initial product design to make it meet the design requirements. Product design optimization is an effective way to improve the quality of a product through minimizing the effect of the causes of variation.⁴⁴ The optimization involves product design optimization and design process optimization. Product design optimization means improving the design in terms of one or more performance aspects of a specific type of product,¹¹¹ while design process optimization could benefit nearly all corresponding product designs produced by adopting the optimized design process.

When a product design is finished, it is expected to maximize performance as well as be less sensitive to variation in practical situations such as environmental changes.^30,44 Design optimization is usually achieved by mathematical optimization techniques.¹¹¹ The mathematical model allows to choose the optimal values of parameters of the model that accurately reflect customers’ expectations.³⁰ In the mathematical model, experiments based on fractional factorial designs and orthogonal arrays can be applied to improve it. However, there are still some challenges in design optimization such as mathematical challenges, topologies and configurations, systems design, controlled artefacts and enterprise-wide design (indicated by Papalambros¹¹¹). Whatever the optimization technique is adopted, the aim is to make the product design as robust as possible. In order to achieve this goal, robust design method can be considered during product design process. In addition, different sources of uncertainties and variations in design and manufacturing process, such as model uncertainty, parameters uncertainty and noise, should be taken into consideration.³⁰

Chen and Li¹¹² proposed a computerized team approach for process design optimization. In their work, they treat design teams as game players in a multi-player game and classify different types of team interactions from the view of game theory. Finally, the satisfaction metric is utilized to evaluate the effectiveness of the proposed approach. Different from their work, Taguchi and Rafanelli¹¹³ divide the design process into three stages – system design, parameter design and tolerance design – and the design focuses of these three stages are basic functional prototype design, controlling parameters to make the design insensitive to variations and the upper limit of the number of variation or noise factors allowed in the design, respectively. Compared to research work in system design, more work is conducted in parameter design and tolerance design.⁴⁴ For example, Cheng and Lin¹¹⁴ took any deviations of design parameters into sensitivity analysis and robust optimization design of suspension system so that the system would have better performance. Liu et al.¹¹⁵ utilized goal programming approach that incorporated analytic network process and cost budget limitation to determine the variant components to be focused on redesign. However, system design determines the attainable level of product robustness in the parameter stage, argued by Andersson.¹¹⁶ Due to the importance of system design, Cheng et al.⁴⁴ proposed a frame on the basis of system modelling, cluster analysis and design of experiments for the development of robust system.

Quality assurance policies

Quality assurance policies are adopted to satisfy the customers by supplying products that fully comply with customer, statutory and regulatory requirements. Most enterprises have their quality assurance departments to ensure the quality of their products and they have their own product or service quality rules such as customer first, priority on quality, safety and confidence and global quality accreditations.⁷⁸

In a questionnaire survey conducted by Dale and Duncalf,⁸⁰ they found that the majority of companies have quality assurance policies, and most of them make their policies in the form of written documents and statements, although some companies utilize verbal communications via the management structure to communicate quality policy information. The attainment level of quality objectives relies on constant examination, improvement and effective communications, and the way of quality assurance policies affects the communications and decision-making processes.⁸⁰ In order to obtain the correct quality products, all participants in the production process should be well controlled and should be motivated into the process. Therefore, operator control method can be combined with job satisfaction schemes, bonus schemes, suggestion schemes and certification schemes in production.⁸⁰

However, the attainment of the desired quality level not only relies on the production enterprise but also relies on its suppliers.^77,80 In the game-theoretic model proposed by Xiao et al.,⁷⁷ they investigated how an enterprise coordinates the supply chain with a quality assurance policy via a revenue-sharing contract and found that the optimal service quality first decreases and then increases as the defective rate of the final product increases. After the product is launched on the market, the enterprises will provide online service by telephone helplines and online/Internet support or will provide field service for helping customers solve problems in the product use process fast.¹⁴ In addition, they will use many databases and information systems to manage customer feedback, complaints and product problem resolution for future improvement of corresponding products.

Research findings and quality control challenges in crowdsourcing context

In the traditional design environment, it is hard to overcome the shortcomings of SMEs in employees and resources, sometimes in budget. Extending the design environment to crowdsourcing context, these shortcomings may be well addressed by leveraging the globally distributed crowds and the cost-effectiveness of crowdsourcing. Therefore, this section mainly analyses the factors affecting product design quality in crowdsourcing context, discusses quality control models and quality assurance policies and investigates quality control challenges over a crowdsourcing platform.

Research findings of quality control studies in crowdsourcing context

When extending the traditional design environment to the crowdsourcing context, the possible design process over a crowdsourcing platform (see Figure 10) has been explored by Niu et al.²¹ They indicated four challenges (please refer to Niu et al.²¹ for more details) when performing product design tasks in the crowdsourcing context.

During this process, there are more factors affecting product design quality. In the crowdsourcing context, three more factor dimensions can be added, that is, requester, task and platform. Figure 11 shows key factor dimensions and some of the key factors affecting the corresponding dimension. More factors affecting product design quality in terms of key elements of crowdsourcing process are shown in Table 6.

Table 6.

Factors affecting product design quality in terms of key elements of crowdsourcing process.

Key elements of crowdsourcing process	Factors affecting design quality
Requester¹¹⁷	Incentives provision, open call, ethicality provision, privacy provision
Platform¹¹⁷	Crowd-related interaction, crowdsourcer-related interaction, task-related facilities, platform-related facilities
Crowds^117–120	Reputation and expertise, instruction and user interface, user identity, nationality, qualification, job title, entry date, department, location, accomplishment, diversity, unknown-ness, largeness, undefined-ness, suitability, social, learning, financial
Task^117–119	Task definition, user interface, granularity, incentives and compensation policy, instruction, task description, target audience, complexity, type of action, modularization, nature of the reward, type of the reward, latency, submission time, closure time, duration, visibility, confidentiality, human requirement, technical resource

The explanation of each factor dimension is shown as follows:

Crowds: It is one of the key elements of a crowdsourcing process. It is extended from the traditional design team, so it can be treated as a virtual team. In traditional design environments, the capability of their employees is well mastered by their team leader. Thus, the team leader can assign tasks to the most right individuals. While in the crowdsourcing context, the truthfulness and reliability of these information provided by the crowds themselves are usually doubtful. In order to assign the task to right persons, these information must be verified.

Requester: It is one of the key elements of a crowdsourcing process. The requester is responsible for proposing task requirements, incentives, timelines and so on.

Task: The task is proposed by the requester and crowds perform it to achieve expected outcomes. In order to better assign the task to crowds, it usually needs to be decomposed either by the requester or by the crowdsourcing platform runner.

Platform: The crowdsourcing platform is a workplace, which provides interfaces for its users including the crowds and the requester to interact with it. All information related to design process, information management, task, the requester and the crowds are controlled by the crowdsourcing platform.

Design process: In the crowdsourcing context, the product design process is controlled by the crowdsourcing platform. The process consists of many subprocesses such as task decomposition, task assignment, the selection of crowds and the synthesis of task results. In order to ensure the collaboration of crowds, the process should have the capability to involve enough qualified crowds by participation motivations and proper selection rules and support the communication among them.¹²¹

Information management: In the crowdsourcing context, all information including product-, process-, crowds- and tasks-related information are stored in database. The users only need to pay attention to the interactions between them and the crowdsourcing platform. Since the information management is controlled by the crowdsourcing platform, the factors affecting software system quality, such as platform performance and data structure, will influence the crowdsourcing platform quality, thereby influencing the quality of work conducted on the platform.¹²²

As product design activity quality control studies in terms of four aspects, that is, crowds, platform, task and workflow, have been reviewed by Niu et al.,^21,55 here we do not review them repeatedly. Here, we mainly compare traditional design environment and crowdsourcing context in terms of the following aspects influencing design activity quality. The comparison is shown in Table 7.

Table 7.

The comparison of traditional environments and the crowdsourcing context.

Comparison item	Traditional design environment	Crowdsourcing context
Design process	Controlled by team leader	Controlled by the requester and platform
Team management	Controlled by team leader	Controlled by the platform
Information management	Documents, videos and so on	Database
The number of participants	Limited	Unlimited
The qualification of participants	Known	Unknown
Incentive mechanism	Bonus, team-building activities and so on	Reward, enjoyment, reputation and so on
Organization structure of participants	Hierarchical structure and cross-functional organization structure	Hierarchical structure
Task description	Team members can discuss to better understand it	The crowd can interpret it by himself or herself or discuss it with other crowds through communication tools
Task decomposition	Performed by the team leader	Performed by the requester or the platform
Task assignment	Assigned by the team leader	Calculated by the platform
Communication	Regular meeting, workshop	Forum, social medium and related tools provided by the platform

Over the crowdsourcing platform, the design process, team management and information management in traditional design environment are all controlled by the crowdsourcing platform.

Discussion of quality control models and quality assurance policies

Literature^18,21,55 found that the existing crowdsourcing platforms can only partly support PDD activities, and the existing studies about product design quality control are relatively less. In crowdsourcing context, a crowdsourcing process with high quality makes it more likely to achieve better product designs. Therefore, many researchers have devoted in investigating crowdsourcing quality control. For example, Daniel et al.¹²¹ have analysed the quality attributes, assessment techniques and assurance policies in crowdsourcing quality control. They mainly control the quality from the perspective of individual, group and computation. The corresponding assessment methods can be found in Daniel et al.¹²¹ However, when PDD activities are performed in the crowdsourcing context, both the traditional product design quality control approaches and crowdsourcing quality control approaches should be considered. In literature,^21,55 we have reviewed some studies about product design quality control approaches (see Table 8). However, most of them focus on controlling the quality of a specific stage of product design process. The quality control models/approaches through the whole product design stages will be investigated in the future.

Table 8.

Quality control approaches and quality assurance policies of crowdsourcing.^118,121

Assessment perspective	Quality control approaches	Quality assurance policies
Individual	Rating, qualification test, self-assessment, personality test, referrals, expert review, usability check and so on	Improve data quality, select people, incentivize people, train people, improve task design, control execution and so on
Group	Voting, group consensus, output agreement, peer review, feedback aggregation, user study and so on
Computation	Ground truth, outlier analysis, finger printing, achievements, implicit feedback, association analysis, content analysis, transfer learning, collusion detection and so on

As for the crowdsourcing quality assurance policies, Daniel et al.¹²¹ have identified six strategies: (a) improve data quality, (b) select people, (c) incentivize people using extrinsic and intrinsic motivations, (d) train people, (e) improve task design and (f) control execution. These strategies aim to improve quality as first-order goal. In addition, some assessment measures have positive side effects on quality, especially when the assessment object are people. For example, reviews have been found by Zhu et al.¹²³ that it can improve the performance of both crowds and reviewers. And rating the performance of crowds has similar positive side effects.¹²⁴ Together with the product design quality assurance policies in traditional design environments such as job satisfaction schemes, certification schemes, free warranty policy, telephone helplines, remote monitoring and customer first, the crowdsourcing quality assurance strategies should make it possible to obtain high-quality product designs in the context of crowdsourcing.

The quality control challenges over a crowdsourcing-based platform

This section mainly focuses on discussing the challenges of product design quality control over a crowdsourcing-based platform. Based on the previous analysis results, it is found that most of the design work is finished by the collaboration of many designers as collaboration is found to be effective in improving product quality and work efficiency.⁵ However, it is hard for SMEs to organize enough designers within their companies to perform the design task unless outsourcing parts of their work to related companies.¹⁰⁶ In addition, it has been demonstrated that crowdsourcing can enable the scaling up of design and manufacturing operations and improve design performance and quality.^18,125

As a result, a lot of enterprises have utilized crowdsourcing to achieve specific goals such as to increase customer engagement and to choose better ideas. However, there are still some challenges on controlling the quality of generated ideas or solutions when product design activities are performed over a crowdsourcing-based collaborative platform. In our previous work, we have analysed the gaps and challenges in adopting crowdsourcing in PDD process from the perspective of crowdsourcing technologies^21,55 and have indicated the necessity of developing relevant tools to support product design and development activities. Except the problem that the existing crowdsourcing platforms and tools cannot fully support PDD activities, there are still other challenges in controlling the product design quality over such a crowdsourcing platform. The possible challenges include the following.

The management of information.

On a crowdsourcing platform, there are more participants in the product design project than in conventional settings. The participants are globally distributed and have various cultural background and different levels of expertise, which may lead to more product design conflicts.

The representation of product design.

On the crowdsourcing platform, the participants have various levels of design capabilities and they may not know how to present their designs as expected.

The communication of designers.

Effective communication plays an important role on improving product design quality, which can help designers have a better understanding of design requirements and work done by their colleagues.

The protection of intellectual property. In order to support the collaboration of crowds, the platform has to support the information sharing and communication. Since crowdsourcing process is open to the crowds registered on the platform, the intellectual property protection faces more risks than in traditional environment.

Aimed at the above challenges, the following corresponding solutions are proposed:

The platform should be cloud-based. With the support of cloud technology, the distributed participants of product design process can easily get access to product and process-related information to effectively perform design activities.

The platform should provide a presentation tool to guide designers to present their designs and other users can pose queries about the design such as rationale and purpose, or the causality between physical and functional elements.

The platform should provide application programming interfaces (APIs) to common social medium because it not only helps designers to communicate with other designers more freely and in real time but also makes them have a connection with their friends.

The platform should integrate with blockchain technology as it supports the encrypted transmission of information.

As there is no such a crowdsourcing platform that fully supports product design process, these solutions cannot be verified until the platform is developed. However, these solutions would provide some guidelines for developing such a platform in controlling product design quality in the future.

Conclusion

This review article mainly analysed the key factors affecting product design quality both in traditional design environments and in the crowdsourcing context. And based on the analysis results, this article indicated product design quality control challenges over a crowdsourcing-based platform and proposed corresponding solutions.

In order to better understand key factors affecting product design quality, this article first gave a definition and sub-attributes of product design quality. Then Tag Cloud is adopted to analyse the keywords to find out the research focuses in these two design environments. In the traditional design environment, four research focuses including information management, information sharing, quality control approaches and quality assurance policies are found. Through analysis, we found that there are more factors affecting product design quality when the design environment is extended to the crowdsourcing context from the traditional design environment. While in the crowdsourcing context, we mainly analysed the key factors affecting product design quality, compared these two design environments and discussed quality control and assurance policies. Based on that, the article indicated four product design quality control challenges over a crowdsourcing-based collaborative platform and proposed corresponding solutions to these challenges.

If these challenges can be well addressed, SMEs will benefit a lot as they can get access to a large pool of crowds with various skills and experience, which will effectively relieve their pressure resulting from the lack of skilled employees and related resources. The research direction in the near future is to develop such a crowdsourcing platform to support PDD activities. This article can provide some guidelines for the platform development.

Footnotes

Acknowledgements

We thank anonymous reviewers for their helpful comments that help to improve the paper.

Handling Editor: Dengkai Chen

Declaration of conflicting interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This research was supported by Senior Foreign Expert Visiting Project (GDW20185100246) and the China Scholarship Council.

ORCID iD

Xiaojing Niu

References

BusinessDictionary. New product development, http://www.businessdictionary.com/definition/new-product-development.html (accessed 3 April 2018).

Wikipedia. Product design, https://en.wikipedia.org/wiki/Product_design (accessed 1 April 2018).

Anastasia. Product quality, https://www.cleverism.com/lexicon/product-quality/ (2017, accessed 1 April 2018).

Chu

Chen

et al . Research on product design quality control methods based on QFD. In: Proceedings of the 11th international conference on computer-aided industrial design & conceptual design (CAIDCD), Yiwu, China, 17–19 November 2010, pp.35–39. New York: IEEE.

Zhu

Alard

You

et al . Collaboration in the design-manufacturing chain: a key to improve product quality. In: Renko

(ed.) Supply chain management: new perspectives. Rijeka: InTech, 2011, pp.199–214.

Mert

Waltemode

Aurich

Quality assessment of technical product-service systems in the machine tool industry. Procedia CIRP 2014; 16: 253–258.

Pasch

Rybski

Jochem

Empirical study on quality management for product-service systems in industrial environment. Bus Process Manage J 2016; 22: 969–978.

Bikas

Stavropoulos

Chryssolouris

Additive manufacturing methods and modelling approaches: a critical review. Int J Adv Manuf Tech 2016; 83: 389–405.

Neumann

Kolus

Wells

RW.

Human factors in production system design and quality performance – a systematic review. IFAC-PapersOnLine 2016; 49: 1721–1724.

10.

Zhang

Gou

et al . A research review on the key technologies of intelligent design for customized products. Engineering 2017; 3: 631–640.

11.

Gao

Zhang

Ramanujan

et al . The status, challenges, and future of additive manufacturing in engineering. Computer Aided Design 2015; 69: 65–89.

12.

Lombard

Van Waveren

Chan

. Factors affecting quality in a manufacturing environment for a non-repairable product. In: IEEE international conference on industrial engineering and engineering management (IEEM), Bandar Sunway, Malaysia, 9–12 December 2014, pp.137–142. New York: IEEE.

13.

Szwejczewski

Goffin

Anagnostopoulos

Product service systems, after-sales service and new product development. Int J Prod Res 2015; 53: 5334–5353.

14.

Markeset

Kumar

Design and development of product support and maintenance concepts for industrial systems. J Qual Maint Eng 2003; 9: 376–392.

15.

Rolstadaas

Hvolby

Falster

. Review of after-sales service concepts. In: Koch

(ed.) Lean business systems and beyond, Boston, MA: Springer, 2008, pp.383–391.

16.

Takeuchi

Quelch

Quality is more than making a good product. https://hbr.org/1983/07/quality-is-more-than-making-a-good-product (1983, accessed 16 April 2018).

17.

Sustainability exchange. Factors affecting recycling, http://www.sustainabilityexchange.ac.uk/factors_affecting_recycling (accessed 30 August 2018).

18.

Qin

Van der Velde

Chatzakis

et al . Exploring barriers and opportunities in adopting crowdsourcing based new product development in manufacturing SMEs. Chin J Mech Eng 2016; 29: 1052–1066.

19.

Tran

Hasan

Park

JY.

Crowd participation pattern in the phases of a product development process that utilizes crowdsourcing. Ind Eng Manage Syst 2012; 11: 266–275.

20.

Poetz

Schreier

The value of crowdsourcing: can users really compete with professionals in generating new product ideas?

J Prod Innov Manage 2012; 29: 245–256.

21.

Niu

Qin

Vines

et al . Key crowdsourcing technologies for product design and development. Int J Automat Comput. Epub ahead of print 27September2018. DOI: 10.1007/s11633-018-1138-7

22.

Djelassi

Decoopman

Customers’ participation in product development through crowdsourcing: issues and implications. Ind Market Manag 2013; 42: 683–692.

23.

Mladenow

Bauer

Strauss

Social crowd integration in new product development: crowdsourcing communities nourish the open innovation paradigm. Glob J Flex Syst Manage 2014; 15: 77–86.

24.

VEX EDR Curriculum. What is the engineering design process?https://curriculum.vexrobotics.com/curriculum/intro-to-engineering/what-is-the-engineering-design-process.html (accessed 15 August 2018).

25.

Dow

How prototyping practices affect design results. Interactions 2011; 18: 54–59.

26.

Lim

Stolterman

Tenenberg

The anatomy of prototypes: prototypes as filters, prototypes as manifestations of design ideas. ACM T Comput-Hum Int 2008; 15: 7.

27.

Gero

JS.

Design prototypes: a knowledge representation schema for design. AI Magazine 1990; 11: 26.

28.

Zhu

Alard

Schönsleben

Design quality: a key factor to improve the product quality in international production networks. In: Olhager

Persson

(eds) Advances in production management systems. Boston, MA: Springer, 2007, pp.133–141.

29.

Aas

EJ.

Design quality and design efficiency; definitions, metrics and relevant design experiences In: Proceedings of the 1st international symposium on quality electronic design (ISQED), San Jose, CA, 20–22 March 2000, pp.389–394. New York: IEEE.

30.

Mrugalska

Tytyk

Quality control methods for product reliability and safety. Procedia Manuf 2015; 3: 5897–5904.

31.

Salimun

Janom

Arshad

. Quality factors of crowdsourcing system: paper review. In: Proceedings of the 6th control and system graduate research colloquium (ICSGRC), Shah Alam, Malaysia, 10–11 August 2015, pp.82–86. New York: IEEE.

32.

Spacey

16 types of design quality, https://simplicable.com/new/design-quality (2017, accessed 20 April 2018).

33.

Hermans

Liu

Quality management in the new product development: a PPAP approach. Qual Innov Prosper 2013; 17: 37–51.

34.

McLeod

Maslow’s hierarchy of needs, https://www.simplypsychology.org/maslow.html (2017, accessed 3 May 2018).

35.

Philips

Design for emotion to increase user engagement, https://www.toptal.com/designers/product-design/design-for-emotion-to-increase-user-engagement (2017, accessed 5 May 2018).

36.

Bradley

Designing for a hierarchy of needs, https://www.smashingmagazine.com/2010/04/designing-for-a-hierarchy-of-needs/ (2010, accessed 8 May 2018).

37.

Schütte

. Engineering emotional values in product design: Kansei engineering in development. PhD Thesis, Linköping University, Linköping2005.

38.

ISO/IEC 25010:2011. Product quality model.

39.

Shao

Zeng

et al . Research progress analysis of reliability design method based on axiomatic design theory. Procedia CIRP 2016; 53: 107–112.

40.

Lutters

van Houten

Bernard

et al . Tools and techniques for product design. CIRP Ann-Manuf Tech 2014; 63: 607–630.

41.

Boess

Experiencing product use in product design. In: Proceedings of international conference on engineering design (ICED), Palo Alto, CA, 24–27 August 2009, pp.311–322. UK: The Design Society.

42.

Sanchez

Inc

Pan

Obtaining reliability insights during a product’s conceptual design process through Bayesian network modelling. Ind Eng Manage 2017; 6: 224.

43.

Mamtani

Green

McDonald

Relative reliability risk assessment applied to original designs during conceptual design phase. Proc IMechE, Part B: J Engineering Manufacture 2006; 220: 917–927.

44.

Cheng

Zhang

Wang

Modelling and analysis of system robustness for mechanical product based on axiomatic design and fuzzy clustering algorithm. Adv Mech Eng 2015; 7: 1–14.

45.

Haug

Münster

. A classification of argument types for product aesthetics. In: Proceedings of the 9th ACM conference on creativity & cognition, Sydney, NSW, Australia, 17–20 June 2013, pp.42–52. New York: ACM Press.

46.

Van Eijk

van Kuijk

Hoolhorst

et al . Design for usability: practice-oriented research for user-centered product design. Work 2012; 41: 1008–1015.

47.

Zaharias

Poylymenakou

Developing a usability evaluation method for e-learning applications: beyond functional usability. Int J Hum-Comput Int 2009; 25: 75–98.

48.

Han

Yun

Kim

et al . Evaluation of product usability: development and validation of usability dimensions and design elements based on empirical models. Int J Ind Ergo 2000; 26: 477–488.

49.

Jovanovic

Starcevic

Jovanovic

. Formal specification of usability measures in model-driven development of context-sensitive user interfaces. In: Proceedings of the international working conference on advanced visual interfaces, Capri Island, 21–25 May 2012, pp.749–752. New York: ACM Press.

50.

Camba

Contero

Parametric CAD modeling: an analysis of strategies for design reusability. Computer Aided Design 2016; 74: 18–31.

51.

González-Lluch

Company

Contero

et al . A survey on 3D CAD model quality assurance and testing tools. Computer Aided Design 2017; 83: 64–79.

52.

Asawa

Goto

Kanazawa

Kansei quality control in product development. FUJITSU Sci Tech J 2009; 45: 179–186.

53.

Özcan

Cupchik

Schifferstein

HN.

Auditory and visual contributions to affective product quality. Int J Design 2017; 11: 35–50.

54.

Vassiliki

Tsakalidis

Tzimas

. Mining interaction patterns in the design of web applications for improving user experience. In: Proceedings of the 27th ACM conference on hypertext and social media, Halifax Regional Municipality, NS, Canada, 10–13 July 2016, pp. 219–224. New York: ACM Press.

55.

Niu

Qin

. A review of crowdsourcing technology for product design and development. In: Proceedings of the 23rd international conference on automation and computing (ICAC), Huddersfield, 7–8 September 2017, pp.1–6. New York: IEEE.

56.

González

FJM

Palacios

TMB

. The effect of new product development technique on new product success in Spanish firms. Ind Market Manag 2002; 31: 261–271.

57.

Frenz

Relevance of workflow analysis and organizational structure, https://smallbusiness.chron.com/relevance-workflow-analysis-organizational-structure-20189.html (accessed 23 June 2018).

58.

Qiu

Chui

Helander

MG.

Knowledge identification and management in product design. J Knowl Manage 2006; 10: 50–63.

59.

Sobek

Jain

VK.

Process factors affecting design quality: a virtual design of experiments approach. J Mech Des 2007; 129: 483–490.

60.

Adams

Turns

Atman

CJ.

Educating effective engineering designers: the role of reflective practice. Design Stud 2003; 24: 275–294.

61.

Atman

Chimka

Bursic

et al . A comparison of freshman and senior engineering design processes. Design Stud 1999; 20: 131–152.

62.

Cross

Expertise in design: an overview. Design Stud 2004; 25: 427–441.

63.

Badke-Schaub

Frankenberger

Analysis of design projects. Design Stud 1999; 20: 465–480.

64.

Pahl

Badke-Schaub

Frankenberger

Resume of 12 years interdisciplinary empirical studies of engineering design in Germany. Design Stud 1999; 20: 481–494.

65.

Kim

SR.

Idea management: identifying the factors that contribute to uncertainty in idea generation practices within front end NPD. Des J 2017; 20: S4398–S4408.

66.

Holladay

Achieving quality through teamwork. Thomasville, NC: Holladay Management Services, 2000, pp.1–5.

67.

Barczak

Wilemon

Factors influencing product development team satisfaction. Euro J Innov Manage 2001; 4: 32–36.

68.

Pimchangthong

Tinprapa

Factors influencing knowledge management process model: a case study of manufacturing industry in Thailand. World Acad Sci Eng Technol 2012; 64: 588–591.

69.

Cummings

Teng

BS.

Transferring R&D knowledge: the key factors affecting knowledge transfer success. J Eng Tech Manage 2003; 20: 39–68.

70.

Kazuhiro

Yamada

Takahashi

et al . A quality engineering approach to human factors affecting software reliability in design process. Electron Comm Jpn 3 2002; 85: 33–42.

71.

Yana

AGA

Rusdhi

Wibowo

MA.

Analysis of factors affecting design changes in construction project with Partial Least Square (PLS). Procedia Eng 2015; 125: 40–45.

72.

Teh

Adebanjo

Ahmed

. Factors affecting product quality and reliability: a comparison of developed and developing countries. In: Proceedings of the IEEE international conference on industrial engineering and engineering management (IEEM), Bandar Sunway, Malaysia, 9–12 December 2014, pp.1481–1485. New York: IEEE.

73.

Wallace

Reid

Clinciu

et al . Culture and the importance of usability attributes. Inform Technol Peopl 2013; 26: 77–93.

74.

Van der Bijl-Brouwer

Van der Voort

. Understanding design for dynamic and diverse use situations. Int J Design 2014; 8: 29–42.

75.

Ding

YF.

Product maintainability design method and support tool based on feature model. J Softw Eng Appl 2009; 2: 165–172.

76.

Tang

Wang

SC.

A systematic methodology for quality control in the product development process. Int J Prod Res 2007; 45: 1561–1576.

77.

Xiao

Yang

Shen

HC.

Coordinating a supply chain with a quality assurance policy via a revenue-sharing contract. Int J Prod Res 2011; 49: 99–120.

78.

GMB. Quality assurance policy, https://www.gmb.jp/en/product/gmb_quality/quality.html (accessed 29 June 2018).

79.

Wikipedia. Quality assurance, https://en.wikipedia.org/wiki/Quality_assurance (accessed 28 June 2018).

80.

Dale

Duncalf

AJ.

A study of quality assurance in small businesses. Proc IMechE, Part B: J Engineering Manufacture 1984; 198: 135–139.

81.

Gao

Bernard

An overview of knowledge sharing in new product development. Int J Adv Manuf Tech 2018; 94: 1545–1550.

82.

Giannini

Monti

Biondi

et al . A modelling tool for the management of product data in a co-design environment. Computer Aided Design 2002; 34: 1063–1073.

83.

Sudarsan

Fenves

Sriram

et al . A product information modeling framework for product lifecycle management. Computer Aided Design 2005; 37: 1399–1411.

84.

Rosen

Wang

et al . Cloud-based design and manufacturing: a new paradigm in digital manufacturing and design innovation. Computer Aided Design 2015; 59: 1–14.

85.

Mourtzis

Vlachou

Zogopoulos

et al . Customer feedback gathering and management tools for product-service system design. Procedia CIRP 2018; 67: 577–582.

86.

Zhang

Shen

Ghenniwa

A review of Internet-based product information sharing and visualization. Comput Ind 2004; 54: 1–15.

87.

Zhang

Zhao

Wang

et al . Research on knowledge support technology for product innovation design based on quality function knowledge deployment. Adv Mech Eng. Epub ahead of print 6June2016. DOI: 10.1177/1687814016651795

88.

Chandrasegaran

Ramani

Sriram

et al . The evolution, challenges, and future of knowledge representation in product design systems. Computer Aided Design 2013; 45: 204–228.

89.

Buskermolen

Terken

Eggen

et al . Effect of visual quality and animation of concept representations on users’ responses to early design concepts: a study on the adaptive patient room concept. Int J Des 2015; 9: 91–106.

90.

Van der Lelie

. The value of storyboards in the product design process. Pers Ubiquit Comput 2006; 10: 159–162.

91.

Deininger

Daly

Sienko

et al . Novice designers’ use of prototypes in engineering design. Design Stud 2017; 51: 25–65.

92.

Van den Hende

Schoormans

Morel

et al . Using early concept narratives to collect valid customer input about breakthrough technologies: the effect of application visualization on transportation. Technol Forecast Soc 2007; 74: 1773–1787.

93.

Culley

Davies

Hicks

et al . An assessment of quality measures for engineering information sources. In: Proceedings of the 15th international conference on engineering design: engineering design and the global economy (ICED 05), Melbourne, VIC, Australia, 1 August 2005, pp.443–456. Barton, ACT, Australia: Engineers Australia.

94.

Anwer

Rashid

Hassan

. Feature based opinion mining of online free format customer reviews using frequency distribution and Bayesian statistics. In: Proceedings of the 6th international conference on networked computing and advanced information management, Seoul, South Korea, 16–18 August 2010, pp.57–62. New York: IEEE.

95.

Yagci

Das

Measuring design-level information quality in online reviews. Electron Commer R A 2018; 30: 102–110.

96.

Ouertani

MZ.

Supporting conflict management in collaborative design: an approach to assess engineering change impacts. Comput Ind 2008; 59: 882–893.

97.

Ouertani

Gzara

Ris

Tracking design dependencies to support conflict management. In: International conference on computer supported cooperative work in design, Nanjing, China, 3–5 May 2006, pp.389–400. Berlin; Heidelberg: Springer.

98.

Barclay

DW.

Interdepartmental conflict in organizational buying: the impact of the organizational context. J Market Res 1991; 28: 145–159.

99.

Cai

Burkett

et al . A methodology for collaborative design process and conflict analysis. CIRP Ann-Manuf Tech 2000; 49: 69–73.

100.

Rose

Gzara

Lombard

Towards a formalization of collaboration entities to manage conflicts appearing in cooperative product design. In: Tichkiewitch

Brissaud

(eds) Methods and tools for co-operative and integrated design. Dordrecht: Springer, 2004, pp.475–486.

101.

Lara

Nof

SY.

Computer-supported conflict resolution for collaborative facility designers. Int J Prod Res 2003; 41: 207–233.

102.

Ouertani

Gzara

Tracking product specification dependencies in collaborative design for conflict management. Computer Aided Design 2008; 40: 828–837.

103.

Yam

Chan

Knowledge sharing, commitment and opportunism in new product development. Int J Oper Prod Man 2015; 35: 1056–1074.

104.

Cummings

JN.

Work groups, structural diversity, and knowledge sharing in a global organization. Manage Sci 2004; 50: 352–364.

105.

Ouertani

Baïna

Gzara

et al . Traceability and management of dispersed product knowledge during design and manufacturing. Computer Aided Design 2011; 43: 546–562.

106.

Mun

Hwang

Han

Protection of intellectual property based on a skeleton model in product design collaboration. Computer Aided Design 2009; 41: 641–648.

107.

Lin

Chen

Lai

HH.

Integrating product information management (PIM) with internet-mediated transactions (IMTs). J Converg Inform Tech 2010; 5: 99–108.

108.

Hasby

Roller

Sharing of ideas in a collaborative CAD for conceptual embodiment design stage. Procedia CIRP 2016; 50: 44–51.

109.

Zhang

Ding

et al . Design change model for effective scheduling change propagation paths. Chin J Mech Eng 2017; 30: 1081–1090.

110.

Zhang

Han

et al . A new conceptual design method to support rapid and effective mapping from product design specification to concept design. Int J Adv Manuf Tech 2016; 87: 2375–2389.

111.

Papalambros

PY.

The optimization paradigm in engineering design: promises and challenges. Computer Aided Design 2002; 34: 939–951.

112.

Chen

A computerized team approach for concurrent product and process design optimization. Computer Aided Design 2002; 34: 57–69.

113.

Taguchi

Rafanelli

AJ.

Taguchi on robust technology development: bringing quality engineering upstream. New York: American Society of Mechanical Engineers, 1994.

114.

Cheng

Lin

YQ.

Multiobjective robust design of the double wishbone suspension system based on particle swarm optimization. Sci World J 2014; 2014: 354857.

115.

Liu

Hsiao

et al . A decision support system for product family design. Inform Sciences 2014; 281: 113–127.

116.

Andersson

On robust design in the conceptual design phase: a qualitative approach. J Eng Design 1997; 8: 75–89.

117.

Hosseini

Phalp

Taylor

et al . The four pillars of crowdsourcing: a reference model. In: Proceedings of the 8th international conference on research challenges in information science (RCIS), Marrakech, Morocco, 28–30 May 2014, pp.1–12. New York: IEEE.

118.

Allahbakhsh

Benatallah

Ignjatovic

et al . Quality control in crowdsourcing systems: issues and directions. IEEE Internet Comput 2013; 17: 76–81.

119.

Finnerty

Kucherbaev

Tranquillini

et al . Keep it simple: reward and task design in crowdsourcing. In: Proceedings of the biannual conference of the Italian chapter of SIGCHI, Trento, 16–20 September 2013, p.14. New York: ACM Press.

120.

André

Kittur

et al . A comparison of social, learning, and financial strategies on crowd engagement and output quality. In: Proceedings of the 17th ACM conference on computer supported cooperative work & social computing, Baltimore, MD, 15–19 February 2014, pp.967–978. New York: ACM Press.

121.

Daniel

Kucherbaev

Cappiello

et al . Quality control in crowdsourcing: a survey of quality attributes, assessment techniques, and assurance actions. ACM Comput Surv 2018; 51: 7.

122.

Sedighi

Zand

. Knowledge management: review of the critical success factors and development of a conceptual classification model. In: Proceedings of the 10th international conference on ICT and knowledge engineering, Bangkok, Thailand, 21–23 November 2012, pp.1–9. New York: IEEE.

123.

Zhu

Dow

Kraut

et al . Reviewing versus doing: learning and performance in crowd assessment. In: Proceedings of the 17th ACM conference on computer supported cooperative work & social computing, Baltimore, MD, 15–19 February 2014, pp.1445–1455.

124.

Dow

Kulkarni

Klemmer

et al . Shepherding the crowd yields better work. In: Proceedings of the ACM 2012 conference on computer supported cooperative work, Seattle, WA, 11–15 February 2012, pp.1013–1022. New York: ACM Press.

125.

Maiolini

Naggi

Crowdsourcing and SMEs: opportunities and challenges. In: D’Atri

Ferrara

George

et al . (eds) Information technology and innovation trends in organizations. Heidelberg: Springer, 2011, pp.399–406.