Development of a university innovation ecosystem assessment model for Association of Southeast Asian Nations universities

Abstract

Universities are centres of innovation and technology transfer, and they play a particularly strong role in the countries of the Association of Southeast Asian Nations (ASEAN) with developing technology programs. However, there are only a few assessment tools for evaluating a university’s innovation ecosystem, and these tools are frequently lacking either in terms of process orientation or in other gaps. The objective of this research was to develop a model of an innovation ecosystem for an ASEAN university and to identify the relative importance of each factor. A multi-method approach was used. In the Stage 1 research, a Delphi study of ASEAN-based university innovation experts (n = 40) was conducted. This research resulted in a preliminary assessment tool. This tool was then evaluated through a broader survey of university innovation experts in ASEAN (n = 418). The data were analysed using confirmatory factor analysis. The results identified a total of 21 innovation areas in four role-based domains: Leaders and Governors, Educators, Innovators and Connectors. The resulting assessment instrument is suitable for use in university innovation ecosystem development and policy making.

Keywords

University innovation ecosystem university assessment entrepreneurial university quality assurance higher education

The university is not just a centre of learning – it is also a network of collaborative relationships between individuals and groups, with links to other universities, industries and government, making it ideal as an environment for innovation (Celuch et al., 2017). Furthermore, the university benefits materially from innovation, with intellectual property (IP) contributing directly to its economic stability and the use of university-developed knowledge and technology contributing to broader economic development (Galan-Muros and Davey, 2019; Jackson, 2011; Perkmann et al., 2011). Despite the importance of innovation in the university and the university in the national innovation ecosystem, however, no consensus model of innovation has emerged. Unlike other domains of educational quality (Elken and Stensaker, 2018), only a few models have been proposed. Instead, measures tend to be either external and market-oriented – for example, the Times Higher Education (THE) or other university rankings – or internal and designed specifically for the individual university. These measures are not necessarily inadequate, but they often rely heavily on the marketized outcomes of innovation, such as the monetization of IP or grant funding. While they are obvious indicators of a university’s innovation progress, such market-oriented indicators have the effect of obscuring other measures of quality that provide useful information for developing its innovation ecosystem (Bendixen and Jacobsen, 2017). Since many universities in countries in the Association of Southeast Asian Nations (ASEAN) are just beginning integration into their national innovation ecosystem (Afzal et al., 2018; Novela et al., 2021), such outcome-oriented metrics are not necessarily useful for understanding the state of immature innovation ecosystems, as they do not reflect growth in capabilities and progress in the short term or opportunities for improvement.

The objective of this research is to develop an assessment instrument for the university innovation ecosystem that could be used across the ASEAN region, which would reflect not just outcomes but progress. The assessment, which was developed through a formative process using the Delphi method, was then performed through evaluation with university professionals. After this process, the items and factor structure were confirmed.

The entrepreneurial university

The concept of the entrepreneurial university was initially defined in the early 2000s, when Henry Etzkowitz and co-authors began to highlight cases in which universities were increasingly involved in not just knowledge production, but also in knowledge application (Etzkowitz, 2003; Etzkowitz et al., 2004). Initially, Etzkowitz (2003) identified the emergence of an entrepreneurship-like structure among research groups, which were discrete sub-organizations under the broader umbrella of the university with their own objectives, external funding and innovation practices. Thus, these groups could be likened to start-up firms even before the emergence of spin-offs and other vehicles of academic entrepreneurship (Etzkowitz, 2003). Slightly later, it was observed that the structure of academic research was shifting, with increasing focus on application and commercialization of research and a narrow gap between theory and application (Etzkowitz et al., 2004). In this work, Etzkowitz et al. (2004) also identified the growing importance of the university as a central actor in regional networks of innovation and economic development. The case studies of the University of Colorado at Boulder and the State University of New York at Albany showed that entrepreneurial universities were increasingly central to the formation of technology and innovation parks, incubators and other entrepreneurial activities, as well as to the direct activities of research groups. Over time, the concept of the entrepreneurial university has shifted and become more defined (Etzkowitz, 2013). As Etzkowitz (2013) outlined, while in the early stages the biggest concern was that the commercialization of the research was being conducted without a central direction, attention to entrepreneurship and innovation subsequently became centralized and consolidated. Furthermore, universities began to actively promote and to become involved in technology transfer, economic development and firm formation and to teach principles of innovation and entrepreneurship (Etzkowitz, 2013). Thus, within the space of a decade, the concept of the entrepreneurial university evolved considerably.

Innovation and the university

Universities play multiple roles in society as centres of adult education and research of different kinds. One commonly used framework for the role of universities envisions three distinct central roles: tertiary (undergraduate) education, professional and specialist education and training at the postgraduate level, and the ‘third mission’ of research and academic and scientific training as a means of contributing to wider society (Laredo, 2007). To this list, it is also possible to add lifelong learning, an adult learning process that continues inside and outside formal education (Milić, 2013). Overwhelmingly, the dominant role of the university is educational, and it is crucial as a social institution of education. However, the third mission is of the university is growing, both in administration and in public reputation and life (Compagnucci and Spigarelli, 2020). Innovation and entrepreneurial training, one part of the third mission, is among the fastest-growing areas of involvement and the one which has received the most attention in the academic literature (Compagnucci and Spigarelli, 2020). At the same time, the turn to innovation and entrepreneurship as central concerns of the university has been critiqued on several grounds. First, research excellence is one aspect of the regional impact of a university, but it is not enough – teaching and community work contribute equally or more to its impact (Bonaccorsi, 2017). Universities also tend to over-focus on specific types of innovation (technical and economic innovation), meaning that their research capabilities are under-utilized and they have little impact on social innovation (Cinar and Benneworth, 2021). There is also a serious question as to whether universities should focus on global ranking status (which is heavily dependent on innovation and entrepreneurship rankings), particularly when the institution still has to achieve development in other areas (Tilak, 2016). In this paper, innovation and entrepreneurship capability development are held to be a single aspect of a university’s function; while not more important than teaching and research, this aspect will be newer to many universities and therefore less systematic and institutionalised.

Innovation ecosystems and the university

The concept of the innovation ecosystem is a complex and contested one, and has undergone several different formulations which are sometimes contradictory (Gomesde et al., 2018). For this research, we use the definition of an innovation ecosystem as an interconnected network of actors (individuals, groups and organizations) that cooperate in formal and informal ways in order to create value (Gomesde et al., 2018). The way innovation ecosystems work to create value, specifically, is through innovation, or the development of new products, services, processes, policies, organizations and other ideas (Gault, 2018).

There are several different roles that actors can assume during the development of the ecosystem, including leadership roles (e.g. ecosystem leader), value creation roles (e.g. supplier) and support roles (e.g. expert) (Dedehayir et al., 2018). Each of these roles involves a distinct set of behaviours and activities rather than an explicit position; therefore, they can be modified to meet the specific environment.

The term ‘university innovation ecosystem’ refers, in its primary sense, to the network of relationships within the university that facilitate its innovation activities (McConnell and Cross, 2019). It also refers to the university’s role in wider national and international innovation ecosystems, which are more important for economic development and industrial advancement (Benitez et al., 2020). While the university does play a direct role in the advanced training of scientists and others to promote innovation (Støren and Aamodt, 2010; Vlasov, 2021), its researchers have working relationships with industry and private companies that also engage in innovation and the commercialization of innovations (McConnell and Cross, 2019). Furthermore, they contribute to innovation by small and medium-sized enterprises, which account for significant amounts of innovative output today (Getmantsev et al., 2020). Therefore, the university and its internal actors also make up part of the broader national and international innovation ecosystems.

Prior assessment frameworks for university innovation ecosystems

There have been several previous assessment frameworks for university innovation ecosystems, although no one measure or approach has achieved consensus (Cheng and Shiu, 2015). From an external perspective, international university ranking systems, like the Times Higher Education Rankings, World University Rankings and QS Rankings, include university innovation outputs as components of the overall ranking (Bilton, 2018; Ewalt, 2019; Simpson, 2012) For example, the Rankings use the Sustainable Development Goals (SDGs) as guidance for their assessment approach (Times Higher Education, 2020). The measures used, including publications, patents citing research and university spin-offs, are primarily output measures whose limitations result in poor rankings for Asian universities despite their level of innovation activities (Dhar, 2020). Furthermore, the output orientation means that progress toward development of the innovation ecosystem is not incorporated into such rankings (De Moortel and Crispeels, 2018; Ugnich et al., 2015).

Another approach to university innovation ecosystem measurement is an internal one, which typically considers both process and output measures. One of these measures, the entrepreneurial university model, assesses formal and informal processes and environmental factors that contribute to innovation in the teaching, learning and research domains (Guerrero and Urbano, 2012; Guerrero et al., 2016; Kirby et al., 2011). However, this model does not offer guidance about how to structure or implement a university innovation ecosystem. Another measure, the entrepreneurial university framework (OECD, 2012), makes up for this weakness by offering specific guidance on structuring university innovation ecosystems. It also acknowledges intra-organizational relationships and international collaboration as key aspects of innovation (Brennan et al., 2014; Leydesdorff and Ahrweiler, 2014). However, it lacks the acknowledgement of informal actions that is included in the entrepreneurial university model. Furthermore, both of these models are actor-oriented, meaning that individuals, rather than roles or processes, are the focal point.

In summary, both the extant external and internal approaches to university innovation ecosystem assessment are inadequate for the tasks of evaluating progress toward innovation ecosystem development and actually planning for innovation ecosystems. Furthermore, market-oriented, output-only measures raise the question of whether quality is being assessed in a meaningful way (Bendixen and Jacobsen, 2017). There is also the question of how widespread is the use any of these models. Most were developed from the experience of a single university or a small number of case comparisons (with the exception of the OECD model). However, there is no clear answer regarding the extent to which the models have been employed in assessment or external comparison, and only the OECD (2012) model was directly designed for use in university administration. This raises the question of whether a better approach for internal developmental use could be refined from these existing models.

Developing a new university innovation ecosystem model

There were three considerations for the development of a new university innovation ecosystem model for use in ASEAN. First, the new model should be useful for assessing the university’s current role in innovation and the broader economic development of its host country – for example, the commercialization of IP and technology transfer (De Moortel and Crispeels, 2018; Ugnich et al., 2015). This was a critical consideration because universities play a key role in national innovation ecosystems in ASEAN (Celuch et al., 2017; Novela et al., 2021). This assessment should not just include marketized aspects such as grant funding or IP licensing fees, as these provide only a partial measure of the value of innovation (Bendixen and Jacobsen, 2017). Instead, it should also include measures that assess broader economic, environmental and social sustainability concerns to reflect the broader contribution to the national innovation ecosystem (Vlasov, 2021).

The new model should also be a developmental one, measuring innovation-oriented processes and policies as well as outputs, since development of these processes is essential for the university’s ultimate output (Bittencourt et al., 2019). This was a critical concern, because the assessment model was designed to be used for development of the university innovation ecosystem, not just its output.

Finally, the instrument under development should be reflective of two additional concerns that are often not considered in university innovation models. The measure should include both formal and informal activities, recognizing that informal collaboration and interaction between actors can result in significant innovation outcomes (Kirby et al., 2011). It also needed to acknowledge that university innovation ecosystems include actors both inside and outside the formal organizational boundaries of the university, for example, in links with industry (McConnell and Cross, 2019) and government and international organizations (Beerkens, 2015).

One of the serious issues of model development formed the basis for measurement. The choice of key factors and measurements for university–industry alliances is problematic, as historically there has been little development of formal assessment systems (Perkmann et al., 2011). There is also a growing question of whether performance measures should be objective and quantifiable, or whether there is space for less quantified measures of intangible outcomes (Dobija et al., 2019). As Dobija et al. (2019) point out, performance measurement serves a number of purposes, both rational and symbolic, in the university and not all measures are appropriate for all purposes. At the same time, it would be inadequate to rely entirely on qualitative measures, because of the potential for bias that is involved in subjective or qualitative performance metrics (Dubey et al., 2017). Therefore, there was a need to carefully consider the measurement model, including what it would measure, how and why.

To begin developing it, the researcher employed the Entrepreneurial University Model. This model (Figure 1), developed for the assessment of universities in Thailand, is role-based and identifies five roles in the university innovation ecosystem and their broad responsibilities. Elaboration of the conceptual model was therefore conducted, incorporating perspectives from the Entrepreneurial University Model (Guerrero and Urbano, 2012; Guerrero et al., 2016; Kirby et al., 2011) and the entrepreneurial university framework (OECD, 2012). The conceptual model (Figure 2), called the ASEAN University Innovation Ecosystem Assessment (AUIEA) model, formed the basis for the model development work. This model incorporates four of the five roles of the Entrepreneurial University Model; the Agent of Change role was eliminated because change is an inherent part of the other four roles.

Materials and methods

The research method was a two-stage study, incorporating multiple research methods. The research process began with a small-scale Delphi study to develop the AUIEA model, and then proceeded to a larger quantitative survey to assess the developed model.

Stage 1: Delphi study

The Delphi method is an approach that uses repeated surveys of subject matter experts to establish consensus on a specific topic (Galanis, 2018). The purpose of the method, which is frequently used in exploratory and future-scanning research, is to incorporate the viewpoint of experts on something that cannot be conclusively proved (Gordon, 2009). Typically, a Delphi study is conducted as a small-scale quantitative study, using repeated rounds of surveys until the question has been answered adequately (Galanis, 2018).

This research used a panel of 40 experts for the Delphi surveys. An expert, for the purposes of the study, was defined as an individual who played a central role in university-based innovation at an ASEAN university. These experts were selected using a purposive and quota sampling technique to ensure participation from known experts in the field of interest and to make sure that all ASEAN countries were represented in the sample. This approach is consistent with sampling for a Delphi study (Galanis, 2018). The sample was made up of participants from all four roles in the model, including policy makers, educators, researchers and commercialization experts. They were recruited through the ASEAN University Network (AUN) and there were at least two experts from each ASEAN country to ensure geographical coverage.

The Delphi study used three rounds of surveys. For each survey, the participants were presented with a series of assessment measures for the university. They were then asked two questions about each measure. First, how relevant was it to university innovation? This was assessed using a 9-point Likert scale (1 = irrelevant, 9 = most relevant) following standard practice (Galanis, 2018). Second, under what dimension of the university innovation ecosystem did it belong? For this question, in the first round participants were asked to freely place the item under one of the main roles. In the second round, they were asked to place the item under a more specific role (e.g. Leaders and Governors/Vision). The majority placement was then confirmed in Round 3. Experts also had the opportunity to provide feedback on each item and the survey in Rounds 1 and 2 – for example, to improve the wording of items or to suggest missing items.

After the first round of the Delphi survey, items with a low consensus (mean < 5) were removed and other items were assigned to one of the dimensional roles based on consensus. Changes were made based on the recommendations, including the addition of new items to round 2. The process was repeated. For the final round, participants were asked to confirm final placement and roles. At this stage, low-consensus items (mean < 7.5) were removed.

Stage 2: Expert survey

Following the Delphi survey, the structure of the draft AUIEA model was assessed using a broader survey of university experts. The sample (n = 418) was selected using a network sampling strategy, in which participants were asked to pass the survey to members of their professional network (Heckathorn and Cameron, 2017). The network sampling approach was chosen because it increases the randomness of the sample and broadens its reach from the initial participants, who may be selected from a small group (Heckathorn and Cameron, 2017). The sample size exceeded the minimum sample size for SEM-based research (n = 200) (Westland, 2010).

The sample included university leaders and innovation policymakers, educators, researchers, innovation coordinators and private-sector innovation partners from around ASEAN. Participant groups included Brunei (n = 20), Cambodia (20), Indonesia (35), Laos (20), Malaysia (39), Myanmar (21), Philippines (28), Singapore (44), Thailand (171) and Vietnam (20).

The survey was presented as an assessment of an ideal university, with the assessment items phrased as ‘The innovative university should…’. These items were assessed on a five-point Likert scale (1 = strongly disagree, 5 = strongly agree). The approach of using the ideal university was intended to control participant bias; since the participants were drawn from universities, their own beliefs and interest in their university rankings could create bias if they were asked to rank their own universities. This does not entirely eliminate participant bias, as participants will still have their own opinions, but, as the study is intended to measure expert opinion and consensus, this is acceptable.

Data from the questionnaire were analysed using confirmatory factor analysis (CFA) to assess the internal structure of the four individual measures and their sub-dimensions. The CFA process was selected because it is useful for assessing measurement models, including model reduction tasks and investigation of model structure (Brown, 2015). Four CFA models were constructed, representing the Leaders and Governors, Educators, Innovators and Connectors dimensions individually. Standard measures for goodness of fit were used, including CFA and TLI (≥0.90) and RMSEA (<0.06) (Hu and Bentler, 1999; Kenny et al., 2015). Model reliability and validity were assessed using standard measures including CR (>0.70), AVE (>0.5) and MSV (<AVE) (Hair, 2016). Finally, the factor loadings were used to determine whether the model structure was adequate, with a lower bound of 0.60 to consider removing or reassigning items (Brown, 2015).

Results

Stage 1: Delphi study

The purpose of the Delphi study was to evaluate two aspects of the AUIEA model: the assessment items included and the dimensional structure of the model. Summary results from the model are presented in Table 1. As these results illustrate, the number of items was reduced significantly between Round 1 and Round 2, but the items did not change much after Round 2. The dimensional structure also did not change significantly after Round 1, as most of the highly ambiguous items were also eliminated from the questionnaire due to non-consensus. The resulting draft assessment, which was used in Stage 2 of the study, is summarized in Table 2.

Table 1.

Summary results of the Delphi study.

	Round 1: Exploration	Round 2: Development	Round 3: Confirmation
Items proposed	135	115	93
Items added	28		0
Items removed	48	24	0
Items retained	115	91	91
Items assigned to…
Leaders:	31	29	29
Vision (V)		5	5
Policies and Strategies (PS)		11	11
Governance and Culture (GC)		3	3
Resource Management (RM)		5	5
Stakeholder Engagement (SE)		5	5
Educators:	24	20	20
Curriculum and Teaching (CT)		10	10
Learning Outcomes (LO)		7	7
Industry Involvement (II)		3	3
Innovators:	33	27	27
Production (IP)		3	3
Commercialization (IC)		3	3
Funding and Financial Management (FFM)		6	6
Incentive and Reward Systems (IRS)		3	3
Training and Mentoring (TM)		3	3
Role Models (RM)		3	3
Business and Innovation Development (BID)		3	3
Faculty Involvement (FI)		3	3
Connectors:	16	15	15
External collaboration (EC)		3	3
Internal collaboration (IC)		3	3
Industry connections (ICO)		3	3
Entrepreneurial Education (EE)		3	3
Entrepreneurial Hub (EH)		3	3

Table 2.

Summary of draft assessment developed from Delphi study.

Scale	Items	Sample items
Leaders
Vision (V)	5	V1. University vision is oriented to innovation and entrepreneurship.
Policies and Strategies (PS)	11	PS1. University policy is oriented to innovation and entrepreneurship.
Governance and Culture (GC)	3	GC1. The university has a flexible organizational structure.
Resource Management (LRM)	5	RM1. University staffing is oriented toward innovation and entrepreneurship.
Stakeholder Engagement (SE)	5	SE1. Internal and external stakeholders in innovation activities are identified at the university level.
Educators
Curriculum and Teaching (CT)	10	CT1. Educators have positive attitudes toward innovation.
Learning Outcomes (LO)	7	LO1. Student enrolment in innovation courses meets or exceeds targets.
Industry Involvement (II)	3	II1. Number of educator links with industry meets or exceeds targets.
Innovators
Production (IP)	3	IP1. Rate of scientific paper publication meets or exceeds targets.
Commercialization (IC)	3	IC1. Sales of patents generated from innovative activities meets or exceeds targets.
Funding and Financial Management (FFM)	3	FFM1. Funding from private industry meets or exceeds targets.
Incentive and Reward Systems (IRS)	6	IRS1. The university has reward systems that offer meaningful incentives for innovation and entrepreneurship.
Training and Mentoring (TM)	3	TM1. Training in innovation and entrepreneurship is available to students and researchers at all levels and disciplines.
Role Models (RM)	3	RM1. Innovation and entrepreneurship role models are visible in the university.
Business and Innovation Development (BID)	3	BID1. Number of start-ups/spin-offs created meets or exceeds targets.
Faculty Involvement (FI)	3	FI1. Number of faculty engaged in research meets or exceeds targets.
Connectors
External Collaboration (EC)	3	EC1. Researchers and educators maintain links with colleagues at other institutions, private industry and other organizations.
Internal Collaboration (INTC)	3	IC1. Researchers and educators across the university engage in cross-disciplinary collaboration.
Industry Connections (ICO)	3	ICO1. The university has inadequate links with industry. (-)
Entrepreneurial Education (EE)	3	EE1. The number of courses addressing entrepreneurship meets or exceeds targets.
Entrepreneurial Hub (EH)	3	EH1. The university has a centralized office or hub for innovation and entrepreneurship coordination.

Stage 2: Factor analysis

The factor analysis investigated the factor structures of the four individual dimensions of the AUIEA model, including Leaders and Governors, Educators, Innovators and Connectors (see Table 3). Goodness of fit is summarized in Table 4, while the factor loadings are summarized in Table 3.

Table 3.

Summary of factor structures.

Construct	Items	α	Items	Initial factor loading	Final model factor loading
Leaders and Governors
V	5	0.808	V1	0.758	0.774
			V2	0.697	0.732
			V3	0.731	0.733
			V4	0.650	0.616
			V5	0.557*	—
PS	11	0.841	PS1	0.432*	—
			PS2	0.434*	—
			PS3	0.424*	—
			PS4	0.643	0.696
			PS5	0.674	0.746
			PS6	0.757	0.788
			PS7	0.650	0.701
			PS8	0.557*	—
			PS9	0.570*	—
			PS10	0.502*	—
			PS11	0.573*	—
			FFM1	0.730	0.730
			FFM2	0.726	0.732
			FFM3	0.712	0.707
IRS	6	0.792	IRS1	0.503*	—
			IRS2	0.601	—
			IRS3	0.605	—
			IRS4	0.619	0.675
			IRS5	0.654	0.725
			IRS6	0.658	0.659
			IRS7	0.526*	—
			BID1	0.677	0.683
			BID2	0.691	0.692
			BID3	0.618	0.612
FI	3	0.780	FI1	0.747	0.817
			FI2	0.914	0.837
			FI3	0.585*	—
Connectors
EC	3	0.688	EC1	0.602	0.613
			EC2	0.685	0.699
			EC3	0.667	0.646
INTC	3	0.706	INTC1	0.567	—
			INTC2	0.798	0.827
			INTC3	0.724	0.764
ICO	3	0.675	ICO1	0.549*	—
			ICO2	0.675	0.692
			ICO3	0.717	0.735
EE	3	0.758	EE1	0.724	0.708
			EE2	0.742	0.740
			EE3	0.682	0.699
EH	3	0.596	EH1	0.636	0.706
			EH2	0.639	0.655
			EH3	0.446*	—

Note: * Factor loading < 0.60, item eliminated during final round.

Table 4.

Goodness of fit measures (final model).

Model	CFI	TLI	RMSEA
Leaders and Governors	0.938	0.922	0.054
Educators	0.939	0.908	0.067
Innovators	0.916	0.893	0.054
Connectors	0.953	0.929	0.047

Leaders and governors

The Leaders and Governors model (Figure 3) included five latent variables. The initial CFA process identified 13 of 29 variables with inadequate factor loadings (<0.60). After these items had been removed, the goodness of fit of the model was adequate (CFI = 0.938, TLI = 0.922, RMSEA = 0.054). The final structure of Leaders and Governors included 16 items, including Vision (V) (4 items), Policies and Strategies (PS) (4 items), Governance and Culture (GC) (2 items), Resource Management (LRM) (3 items) and Stakeholder Engagement (SE) (4 items). Of these factors, the factor loading was highest for V (0.865), followed by SE (0.712), GC (0.708), PS (0.621) and LRM (0.519). All of these factors exceeded the factor loading of 0.40, which is a conservative measure of inclusion for a given scale (Brown, 2015).

Educators

The Educators model (Figure 4) included three latent variables. Initially, this included 20 items, but 8 were removed due to factor loadings < 0.60 in the initial CFA round. The final Educators model included Curriculum and Teaching (CT) (5 items), Learning Outcomes (LO) (5 items) and Industry Involvement (II) (2 items). Goodness of fit for the final Educators model also passed the minimum threshold (CFI = 0.939, TLI = 0.908, RMSEA = 0.057). The highest factor loading was observed for CT (0.906), followed by LO (0.748) and II (0.626).

Innovators

The Innovators model (Figure 5) consisted of 8 latent variables. Initially, this included 27 items, with 6 items removed during the initial CFA round. The final model included the constructs of Production (IP) (2 items), Commercialization (IC) (3 items), Funding and Financial Management (FFM) (3 items), Incentive and Reward Systems (IRS) (3 items), Training and Mentoring (TM) (3 items), Role Models (RM) (3 items), Business and Innovation Development (BID) (3 items) and Faculty Involvement (FI) (2 items). The model was adequately fitted according to the goodness of fit measures (CFI = 0.916, TLI = 0.893, RMSEA = 0.054). The TLI measure is slightly lower than would be ideal, but this could not easily be improved. The highest factor loading was observed for IP (0.821), followed by FFM (0.748), FI (0.730), BID (0.712), RM (0.688), IC (0.656), TM (0.639) and IRS (0.582).

Connectors

The Connectors model (Figure 6) included five latent constructs, with a total of 15 items in the first round of CFA. This was reduced with the removal of 3 items, for a total of 12. The resulting goodness of fit for this model was acceptable (CFI = 0.953, TLI = 0.929, RMSEA = 0.047). The constructs included External Collaboration (EC) (3 items), Internal Collaboration (INTC) (2 items), Industry Connections (ICO) (2 items), Entrepreneurial Education (EE) (3 items) and Entrepreneurial Hub (EH) (2 items). The strongest factor loading for this scale was EC (0.761), followed by ICO (0.748), EE (0.697), INTC (0.650) and EH (0.601).

Conceptual framework

Following the completion of the CFA process, the most important factors identified were used to refine the conceptual framework presented in Figure 2. This refined model (Figure 7) incorporates the significant factors in each component. This is a significant shift from the initial model, as it shows the added, removed and changed dimensions of the leadership model. While this model does include all factors that had a factor loading of above 0.40, it is clear that some of the factors had a higher impact than others. Factors including LRM (in Leaders and Governors) and IRS (in Innovators), while above the minimum value of 0.40, did fall below the inclusion threshold of 0.60 which was used for the individual items that were loaded onto the factors. There is no strict guide as to when something should be included in CFA (Brown, 2015), but the results may indicate that these factors are somewhat weaker than the others. This question will be addressed in future testing of the model.

Discussion and conclusion

The AUIEA model is an integrative model which incorporates the strengths of existing university innovation assessment models while compensating for their weaknesses. The preliminary model as developed combined aspects including the formal and informal perspectives on innovation of the entrepreneurial innovation model (Guerrero and Urbano, 2012; Guerrero et al., 2016; Kirby et al., 2011); the acknowledgement of intra-organizational and inter-organizational collaborative relationships and the international orientation of the entrepreneurial university framework (OECD, 2012); and the role-oriented perspective and dimensions of the Entrepreneurial University Model (Thawesaengskulthai, 2018; 2020). At the same time, the AUIEA model is not strictly defined or limited by any of these models. Instead, it includes a broader perspective on university innovation, driven by concerns such as the need to consider the processes and policies of the innovative university, not just its outcomes. While there are some similarities in the resulting measure with externally oriented, outcome-based independent measures like the Times Higher Education Rankings, World University Rankings, and QS Rankings (Bilton, 2018; Ewalt, 2019; Simpson, 2012), these were not the main focus of the findings for a simple reason: the external, market-oriented perspective on innovation encompassed in metrics like publication rates or patent filings is measuring only the visible and tangible outcomes of university innovation ecosystems, not the internal processes and less-tangible informal and cultural aspects of innovation. Of course, these internal processes and cultural aspects are also inadequate on their own as they are subject to bias, particularly if measured only by interested stakeholders (such as university administrators). Thus, the combination of externally comparable and internally assessed quantitative and qualitative measures offsets the weaknesses of each measure, providing a holistic view of the overall state of the innovation ecosystem at the university.

In conclusion, this research has achieved the initial development of a hybrid model for university innovation ecosystem assessment. It is intended as a developmental model, rather than a ranking model. This means that, rather than being used for external rankings as in the Times Higher Education (2020) and other external measures, it is intended for use internally in the university system to establish the current position of a university’s innovation ecosystem and plans for further development. Thus, it is unlike any of the existing models that currently are used to assess university innovativeness.

What the AUIEA model demonstrates most effectively is that the university innovation ecosystem is remarkably complex, and has many associated actors, processes and outcomes. As for any complex system, its measurement is also complex. Furthermore, it is not clear that all universities can or should have the same goals for innovation ecosystem development. For example, some with mature innovation ecosystems may prioritize outputs like IP production and licensing, while universities just beginning to develop an innovation ecosystem may focus more on establishing policies, culture and systems that will promote positive attitudes and innovation practices. It is expected that in ASEAN, which includes both countries such as Thailand and Singapore with mature innovation systems and countries like Myanmar where innovation systems are just developing (Novela et al., 2021), there will be a need for universities to set their own priorities and prioritize the process and output measures that make sense for their stage of development.

This model is important for universities in ASEAN in the long term because it can help them proactively adapt to disruption in the university sector. With an increasing focus on innovation and the role of the university as the centre of networks of innovation and entrepreneurship, many universities may struggle to position themselves for their changing roles in society. By developing a university innovation ecosystem, a university can adapt to these changes and thrive in an increasingly globalized education market, while drawing on new resources and partners to assist it. The AIUEA assessment model is one tool that can help universities get to this position.

There are some practical implications for university management. This research has provided universities in ASEAN with a practical tool that can be used for assessment of the university innovation ecosystem and identification and prioritization of areas for improvement. However, the tool does need to be incorporated into strategic and policy planning in order to be effective. Thus, the AUIEA model is not enough on its own to promote university-based innovation or engagement with broader networks of innovators. This is a serious concern for some universities where the innovation ecosystem model is not yet in use, and should be considered when implementing the measurement model.

It is important to note that this model will not in and of itself make a university more innovative. Although its application is still in development, the intended use is as an internal, formative model applied to assess the current state of the university’s innovation processes and systems. This will give university administrators insight into where there are problems and what improvements could be made, but these improvements will then need to be implemented in strategic and operational decisions. However, the combination of quantitative and qualitative measures means that there are limitations on how well the model could be used for external comparison. Furthermore, if the AIUEA model is used for internal assessment, it will be best applied by independent auditors or assessors rather than used by the university’s administration. This will provide the best assessment, avoiding the unintentional bias that results from assessment by interested stakeholders. This possibility for unintentional bias is one of the inherent weaknesses of the model, which will be addressed in future refinements.

There are some limitations to this research. The most significant limitation is that it has not yet been developed into a maturity model that universities could use for the prioritization task above. This is an obvious opportunity for further improvement, and one which the researcher is currently engaging in through a process of further expert surveys. The outcome of the ongoing research will be a capability maturity model for university innovation ecosystems, which will help universities to assess their maturity levels and establish goals and priorities for investment.

Footnotes

Declaration of conflicting interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) received no financial support for the research, authorship, and/or publication of this article.

ORCID iD

Jirawan Chaipongpati

References

Afzal

MNI

Sulong

Dutta

Mansur

, et al. (2018) ‘An investigation on triple helix model and national innovation systems: The case of Malaysia’. Journal of Entrepreneurship Education 21(2): 1–15.

Beerkens

(2015) ‘Quality assurance in the political context: in the midst of different expectations and conflicting goals’. Quality in Higher Education 21(3): 231–250. DOI: 10.1080/13538322.2015.1111004.

Bendixen

Jacobsen

(2017) ‘Nullifying quality: The marketisation of higher education’. Quality in Higher Education 23(1): 20–34. DOI: 10.1080/13538322.2017.1294406.

Benitez

Ayala

Frank

(2020) ‘Industry 4.0 innovation ecosystems: An evolutionary perspective on value cocreation’. International Journal of Production Economics 228: 107735. DOI: 10.1016/j.ijpe.2020.107735.

Bilton

(2018) Rankings Explained: Are university rankings actually relevant to students? Study International.

Bittencourt

Carneiro Zen

Gazaro dos Santos

(2019) A‘Orchestrating university innovation ecosystem: The case of a Brazilian universtiy’. Revue Internationale d’Intelligence Économique II(June): 69–95. Available at: https://www.cairn-int.info/revue-internationale-d-intelligence-economique-2019-2-page-69.htm (Accessed 15 March 2020).

Bonaccorsi

(2017) ‘Addressing the disenchantment: universities and regional development in peripheral regions’. Journal of Economic Policy Reform 20(4): 293–320. DOI: 10.1080/17487870.2016.1212711.

Brennan

, et al. (2014) Study on Innovation in Higher Education: Final report, European Commission Directorate for Education and Training Study on Innovation in Higher Education . DOI: 10.2766/66003.

Brown

(2015) Confirmatory Factor Analysis for Applied Research. New York: The Guilford Press.

10.

Celuch

Bourdeau

Khayum

, et al. (2017) ‘The role of the university in accelerated learning and innovation as a regional ecosystem integrator’. Journal of Research in Innovative Teaching & Learning 10(1): 34–47. DOI: 10.1108/jrit-10-2016-0009.

11.

Cheng

CCJ

Shiu

(2015) ‘The inconvenient truth of the relationship between open innovation activities and innovation performance’. Management Decision 53(3): 625–647.

12.

Cinar

Benneworth

(2021) ‘Why do universities have little systemic impact with social innovation? An institutional logics perspective’. Growth and Change 52(2): 751–769. DOI: 10.1111/grow.12367.

13.

Compagnucci

Spigarelli

(2020) ‘The third mission of the university: A systematic literature review on potentials and constraints’. Technological Forecasting and Social Change 161(September): 120284. DOI: 10.1016/j.techfore.2020.120284.

14.

Dedehayir

Mäkinen

Roland Ortt

(2018) ‘Roles during innovation ecosystem genesis: A literature review’. Technological Forecasting and Social Change 136: 18–29. DOI: 10.1016/j.techfore.2016.11.028.

15.

Dhar

(2020) Innovation in South and South East Asia. London, UK: Clarivate.

16.

Dobija

Górska

Grossi

, et al. (2019) ‘Rational and symbolic uses of performance measurement: Experiences from polish universities’. Accounting Auditing and Accountability Journal 32(3): 750–781. DOI: 10.1108/AAAJ-08-2017-3106.

17.

Dubey

Gunasekaran

Childe

, et al. (2017) ‘Examining the effect of external pressures and organizational culture on shaping performance measurement systems (PMS) for sustainability benchmarking: Some empirical findings’. International Journal of Production Economics 193: 63–76. DOI: 10.1016/j.ijpe.2017.06.029.

18.

Elken

Stensaker

(2018) ‘Conceptualising “quality work” in higher education’. Quality in Higher Education 24(3): 189–202. DOI: 10.1080/13538322.2018.1554782.

19.

Etzkowitz

(2003) ‘Research groups as “quasi-firms”: The invention of the entrepreneurial university’. Research Policy 32(1): 109–121. DOI: 10.1016/S0048-7333(02)00009-4.

20.

Etzkowitz

(2013) ‘Anatomy of the entrepreneurial university’. Social Science Information 52(3): 486–511. DOI: 10.1177/0539018413485832.

21.

Etzkowitz

Schuler Jnr

Gulbrandsen

(2004) The future of knowlege production in the academy. In: Jacob

Hellström

(eds.), ‘The Evolution of the Entrepreneurial University’. Bucingham, UK: SHRE/Open University Press, pp. 40–60.

22.

Ewalt

(2019) The World’s Most Innovative Universities 2019. London, UK: Reuters.

23.

Galan-Muros

Davey

(2019) The UBC ecosystem: putting together a comprehensive framework for university-business cooperation. Journal of Technology Transfer 44(4): 1311–1346. DOI: 10.1007/s10961-017-9562-3.

24.

Galanis

(2018) ‘The Delphi method’. Archives of Hellenic Medicine 35(4): 564–570. DOI: 10.4324/9781315728513-10.

25.

Gault

(2018) ‘Defining and measuring innovation in all sectors of the economy’. Research Policy 47(3): 617–622. DOI: 10.1016/j.respol.2018.01.007.

26.

Getmantsev

Myasnikova

Ermolenko

, et al. (2020) ‘Competitive advantages of the innovative university ecosystem as the source of the competitiveness of small innovative enterprises*’. Advances in Economics Business and Management Research 110(18): 559–564. DOI: 10.2991/aebmr.k.191225.100.

27.

Gomesde

LAV

Facin

ALF

Salerno

, et al. (2018) ‘Unpacking the innovation ecosystem construct: Evolution, gaps and trends’. Technological Forecasting and Social Change 136: 30–48. DOI: 10.1016/j.techfore.2016.11.009.

28.

Gordon

(2009) ‘Future research methodology’. Future Research Methodology, 29–31.

29.

Guerrero

Urbano

Fayolle

, et al. (2016) ‘Entrepreneurial universities: emerging models in the new social and economic landscape’. Small Business Economics 47(3): 551–563. DOI: 10.1007/s11187-016-9755-4.

30.

Guerrero

Urbano

(2012) ‘The development of an entrepreneurial university’. Journal of Technology Transfer 37(1): 43–74. DOI: 10.1007/s10961-010-9171-x.

31.

Hair

(2016) Multivariate Data Analysis. 7th edn. New York: Pearson.

32.

Heckathorn

Cameron

(2017) ‘Network sampling: From snowball and multiplicity to respondent-driven sampling’. Annual Review of Sociology 43: 101–119. DOI: 10.1146/annurev-soc-060116-053556.

33.

Bentler

(1999) ‘Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives’. Structural Equation Modeling 6(1): 1–55. DOI: 10.1080/10705519909540118.

34.

Jackson

(2011) What is an innovation ecosystem? DOI: 10.1017/CBO9781107415324.004.

(2015) ‘The performance of RMSEA in models with small degrees of freedom’. Sociological Methods and Research 44(3): 486–507. DOI: 10.1177/0049124114543236.

(2011) ‘Making universities more entrepreneurial: Development of a model’. Canadian Journal of Administrative Sciences 28(3): 302–316. DOI: 10.1002/CJAS.220.

37.

Laredo

(2007) ‘Revisiting the third mission of universities: Toward a renewed categorization of university activities?’ Higher Education Policy 20(4): 441–456. DOI: 10.1057/palgrave.hep.8300169.

38.

Leydesdorff

Ahrweiler

(2014) ‘In search of a network theory of innovations: Relations, positions and perspectives’. Journal of the Association for Information Science and Technology 65(11): 2359. DOI: 10.1002/asi.

(2019) ‘Realizing the value of industry-university innovation alliances’. Research Technology Management 62(2): 40–48. DOI: 10.1080/08956308.2019.1563437.

40.

Milić

(2013) ‘The twenty-first century university and the concept of lifelong learning’. Australian Journal of Adult Learning 53(1): 159–179.

41.

De Moortel

Crispeels

(2018) ‘International university-university technology transfer: Strategic management framework’. Technological Forecasting and Social Change 135: 145–155.

, et al. (2021) ‘Creating a university-based entrepreneurial ecosystem in Indonesia’. Academic Journal of Interdisciplinary Studies 10(1): 174–183. DOI: 10.36941/ajis-2021-0015.

43.

OECD (2012) ‘A Guiding Framework for Entrepreneurial Universities’. Paris, France: OECD & European Commission, pp. 1–54.

(2011a) ‘Engaging excellence? Effects of faculty quality on university engagement with industry’. Research Policy 40(4): 539–552.

(2011b) ‘How should firms evaluate success in university-industry alliances? A performance measurement system’. R and D Management 41(2): 202–216. DOI: 10.1111/j.1467-9310.2011.00637.x.

46.

Simpson

(2012) Reputation to consider? Check the university league tables. Dubai: Times Higher Education (THE) World University Rankings.

47.

Støren

Aamodt

(2010) ‘The quality of higher education and employability of graduates’. Quality in Higher Education 16(3): 297–313. DOI: 10.1080/13538322.2010.506726.

48.

Tilak

JBG

(2016) ‘Global rankings, world-class universities and dilemma in higher education policy in India’. Higher Education for the Future 3(2): 126–143. DOI: 10.1177/2347631116648515.

49.

Times Higher Education (2020) Impact Rankings Methodology 2021. Available at: http://ipb.ac.id (Accessed 7 February 2020).

50.

Ugnich

Chernokozov

Velichko

(2015) ‘Innovation ecosystem of higher educational institution as a driver of commercialization of intellectual activity results’. Mediterranean Journal of Social Sciences 6(6): 239–245.

51.

Vlasov

(2021) ‘Higher education system as the driver of innovation development of the country’. Journal of Contemporary Issues in Business and Government 27(1): 1136–1146.

52.

Westland

(2010) ‘Lower bounds on sample size in structural equation modeling’. Electronic Commerce Research and Applications 9(6): 476–487. DOI: 10.1016/j.elerap.2010.07.003.