4 Analysis of STARS as a Sustainability Assessment System Universally Usable in Higher Education a System Higher

Sustainability assessment has become an important tool for measurement of higher education institutions. Although there are many potential tools for measuring an institutions level of sustainability, this paper explores the use of STARS as a system for higher education sustainability that could be used universally by all higher education institutions, as well as enables comparison between institutions. The results of this paper address the effectiveness of STARS to fulfill this need, as well as the further development needed in order to fully encompass sustainability and stakeholder needs.

§ 4.1 Introduction: Over recent years various sustainability assessment systems have been developed for the higher education sector. Both policy makers (UNESCO 2011) and students (Bone and Agombar 2011) have placed a significant focus on sustainability in higher education. Along with the various interpretations of sustainability, higher education institutions have been led worldwide to adopt a wide range of assessment systems to benchmark, report and compare various aspects of institutions' sustainability.
Although all these systems offer a way of quantifying some level of sustainability within higher education, they do not provide a universal template that enables stakeholders such as students, academics and managerial staff, to compare levels of sustainability between institutions. Sustainability has become an increasingly important factor in students decision making and not having a way of comparing sustainability within institutions may lead to erroneous information or poor criteria for selection. For example, there is evidence that sustainability messaging tends, for the greatest part, to treat 'sustainability' as synonymous with 'the environment' (Selby et al. 2009).
Although there is resistance to standardizing assessments and/or rating institutions on sustainability, Maragakis and Dobbelsteen's (2015) research provided justification for the use of AASHE's Sustainability Tracking, Assessment and Rating System (STARS) as a universal sustainability assessment tool. Based on empirical data collected and literature reviews, the STARS system showed that it was the best suited to provide a basis for a sustainability assessment system within higher education institutions that would allow for universal comparison.
Even though the STARS system was shown to be the most suitable of the current available systems, it is not without flaws. Maragakis and Dobbelsteen (2015) showed that there were some lapses within STARS that would need to be addressed in order for the STARS system to truly meet both sustainability assessment criteria. Furthermore, the empirical data collected by Maragakis and Dobbelsteen (2013) also show that STARS overlooks some basic stakeholder needs. This paper explores the weaknesses of the STARS framework identified in these preceding publications. § 4.2 The need for a universal system for higher education sustainability This paper stipulates that an ideal system for higher education sustainability is one that can be used universally by all higher education institutions, as well as enables comparison between institutions.
Within the higher education sector, institutions at a national and international level are ranked and compared with one another on a range characteristics and accomplishments encompassing academic quality, athletics, and even student partying. While this tendency to rate and rank institutions has numerous critics from within and outside the higher education sector, transparent ratings and rankings that are based on measurable and relevant metrics can provide information valuable information to key stakeholders. Most notably, such systems can enable prospective students to choose an institution based on the factors and qualifications that most interest them.
A standardized sustainable assessment system could serve as a standard for sustainability marketing toward prospective students and other stakeholders, assuring that 'sustainability' is not misrepresented as a solely environmental issue while also assisting with the deepening of sustainability within the institutions culture. Maragakis and Dobbelsteen's (2013) empirical study showed that 95% of potential or current students, staff and management in higher education agreed that there was a need for a uniform rating system. This demand would explain the rise of certain private initiatives, such as Princeton's Guide to 311 Green Colleges (The Princeton Review 2011).
In general, there has been resistance to standardized assessments and efforts to rate higher education institutions on sustainability, a situation that arguably neither benefits sustainable practices nor helps stakeholders identify the level of sustainability in an institution. By resisting the development of a standardized system, scholars and practitioners may lose the ability to shape assessment and rating criteria for sustainability and could give rise to popular, yet potentially ineffective, methods of assessment that appeal to institutional stakeholders.
The objective of this paper looks to build upon previous research that has identified lapses within sustainability assessment systems that directly relate to STARS and analyze them. STARS will be scrutinized based on these criteria utilizing the tool itself as well as stakeholder input. The methodology will be to compile the previous research and stakeholder input in order to disseminate the key lapses currently found in STARS. These analyzed lapses will provide concrete information for improvement of the tool and utilization universally amongst institutions. § 4.3 An Overview of STARS STARS has grown to become a widely used international standard in higher education sustainability assessment. STARS was established and is administered by the Association for the Advancement of Sustainability in Higher Education (AASHE), with broad participation from the higher education community. STARS is a voluntary, self-reporting framework developed to recognize and gauge relative progress toward sustainability for colleges and universities. The STARS framework is the result of a 2006 call for a campus sustainability rating system by the Higher Education Associations Sustainability Consortium, which was a collaboration of AASHE and other nonprofit-organizations and higher education institutions (AASHE 2005). The still evolving STARS framework uses generally agreed upon American and international standards as assessment tools to assess and guide universities' decision making concerning sustainability, emphasizing four main categories:

-Education and Research -Operations -Planning, Administration and Engagement -Innovation § 4.4 STARS as an Ideal Sustainability Measurement System
Although there is a multitude of literature and academic conversation on STARS' effectiveness and usability within a diverse range of higher education institutions, few have gone so far as suggesting a course of action to make STARS suitable for use in all institutions. The two research publications by Maragakis and Dobbelsteen have identified STARS as the most appropriate assessment tool for sustainability within higher education. However, in both research pieces serious lapses were identified that limit the STARS system from being an ideal universal system.
Preceding research concluded that STARS was the best suited assessment tool for universal use, based on criteria set forth in previous literature specific to sustainability assessment methods (Maragakis and Dobbelsteen 2015), and the empirical results from a survey conducted in 2012 in which 60% of the participants agreed that STARS was the most suitable assessment tool for assessing institutions. These findings support the idea that STARS is the most popular assessment system for higher education institutions, a conclusion that has been validated in other literature (GreenerU 2010;Saadatian et al. 2011).
An ideal sustainability measurement system for the higher education sector, beyond being universally applicable to all higher education institutions would need to appeal both to the founding principles of sustainability within higher education and the needs of the stakeholders interested in the institutions. STARS provides a methodological step towards sustainability assessment for higher education institutions, however it falls short of the parameters of a measurement systems that is universally applicable to all higher education institutions. § 4.5 Lapses in the STARS system based upon previous literature Building on the findings of preceding publications, this research accepts the assumption that STARS is the most appropriate sustainability assessment tool for universal use. In Maragakis and Dobbelsteen (2015), a detailed review of the literature relevant to sustainability assessment systems in higher education systems revealed eleven criteria to be used to assess the most suitable system for universal use. These criteria, and whether each is addressed by STARS are denoted in Table 4.1.

STARS' COVERAGE OF CORE CAMPUS SUSTAINABILITY ISSUES
Addressed by STARS?

Core issues of ecologically, socially and fiscally sustaining a society and campus (Orr, 2000)
What quantity of material goods does the college/university consume on a per capita basis?

Yes
What are the university/college management policies for materials, waste, recycling, purchasing, landscaping, energy use and building?

Yes
Does the curriculum engender ecological literacy? Yes Do university/college finances help build sustainable regional economies? Partially What do graduates do in the world? No

Identify important issues Yes
Are calculable and comparable Yes Move beyond eco-efficiency Yes

Identifying Strengths and Weakness of Sustainable Higher Educational Assessment Approaches (Saadatian et al. 2011)
Popularity Yes TOC STARS met nine of the eleven criteria set forth in previous literature. STARS does not address what graduates do in the world at all, which takes into account the extent to which graduates still engage in sustainability-related employment or other activities, and it was found to only partially address how college finances help build sustainable regions. An analysis of the lapses identified in Table 4.1 is provided in the following sections. § 4.5.1 Lapse in Regional Economic Assessment With regards to the question, "do university/college finances help build sustainable regional economies?", the STARS system does promote and assess some regional partnerships and initiatives, but falls short of providing concrete methods for higher education institutions' regional integration. This concept is not clearly defined and thus allows room for debate to whether or not the STARS system has fully captured this requirement.
The international study conducted by Puukka (2008) presents a set of criteria to be used to assess the regional impacts of higher education institutions. This study found that higher education institutions' regional engagement is typically related to the following areas: -Contributions of research to regional innovation, -The role of teaching and learning in the development of human capital, -Contributions to social, cultural and environmental development, and -The role of higher education institutions in building regional capacity to act in an increasingly competitive global economy.
The following assesses the level to which STARS adequately addresses these four areas of regional engagement: Contributions of research to regional innovation: STARS includes numerous credits under the innovation category, but does not specifically favor or emphasize innovation that is regional in nature. In addition, credits indirectly dealing with regional innovation and advancement are included throughout the STARS framework. (Partially addressed by STARS) The role of teaching and learning in the development of human capital: STARS provides ample credits related to sustainability teaching and learning. However it should be noted that these credits could be fulfilled through more narrowly focused eco-literacy efforts, and thus may not provide a complete framework for approaching the development of sustainability-literate human capital. This observation warrants further research in order to determine a well-balanced approach for future use. (Fully addressed by STARS) Contributions to social, cultural and environmental development: STARS provides various credits that support both student and institutional involvement in a variety of initiatives that promote integration with regional stakeholder and assist with regional development. (Fully addressed by STARS) Building regional capacity to act in an increasingly competitive global economy: The STARS framework treats economics rather ambiguously. Though economics is often referenced in context with the triple bottom line of society, environment and economy, STARS offers very few credits that directly address this topic. Furthermore, STARS provides no framework for measuring institutions' economic impact on its students. These could include the sustainability of tuition, employability of students postgraduation, opportunity cost of studying. These simple economic indicators indirectly assess the ability for students to help create a more competitive regional capacity.

(Not addressed by STARS)
Overall, the STARS system partially addresses regional issues but falls short of providing a meaningful framework to fully address regional integration. § 4.

Lapse in Post-Graduate Metrics
The most noticeable omission in the STARS system based on the eleven criteria set forth by Maragakis and Dobbelsteen (2015) was in regards to assessing what graduates do in the world.
Prior to assessing this criterion it is important to note that there is no clear definition of what this entails. With regards to sustainability, an assessment of post-graduate impact could encompass a wide variety of parameters that focused on the triple bottom line. These actions could be aligned with measuring the effectiveness of learning outcomes with regards to social, environmental and economic factors, as an example. Irrespective of what this specific criterion entails, without even a basic framework for assessing an institutions' graduates as they develop professionally, there is a clear failure to implement any aspect of this criterion within the current iteration of the STARS framework. The lack of any measure to assess the main product of higher education institutions limits STARS usefulness as a universal assessment tool for sustainability in higher education. § 4.

Lapses in the STARS system based on stakeholder needs
Beyond just the eleven criteria set forth in Table 4.1, there is also a significant lapse in STARS from the perspective of the stakeholders.
The results of Maragakis and Dobbelsteen's (2013) empirical survey indicated that 60% of respondents thought that the STARS system was the most appropriate to assess an institutions level of sustainability. However, as shown below in Figure 4.1, the same respondents also have a high level of dissatisfaction with the ability of STARS to offer an all-encompassing and well-balanced system for measuring sustainability. The response shows that from the 60% of participants that support STARS as the best assessment method for universal use, 72% of them believe that is not a well-balanced system and indicates that there is significant improvement needed within the STARS framework.
Of the respondents that supported STARS as the best system for assessing an institution's sustainability, 69% offered their opinion on which categories of the STARS system needing improvement. These responses are summarized below in  Notably, 46% of the opinion offered indicated that all sections of the STARS system need improvement. Of the responses provided, the relative concentrations of the responses are shown in Figure 4.3 which suggests that the most problematic areas within STARS seem to be the subcategories of innovation and operations. All the respondents were offered the opportunity to offer suggestions for improvement. Of all the respondents, only seven offered more detailed opinions. These responses, edited for grammar, were as follows: -The innovation section needs more information.
-The innovation lacks any sort of fundamental framework -The innovation essential provides nothing noteworthy for progressing a universities path towards sustainability.
-STARS needs to have clear goals for innovation.
-STARS provides very vague guidelines.
-STARS needs to include interim targets, party evaluation and student engagement in each section.
-Operations needs to focus on investment with respects to climate change, fossil fuel companies and carbon risks.
Although these responses do not provide enough information to serve as concrete recommendations for improving STARS, it is noteworthy that the majority of feedback is directed toward the innovation section. These responses, coupled with the high response rate in Figure 3, indicate that innovation seems to be one of the most problematic sub-categories.
Another important result from Maragakis and Dobbelsteen's (2013) empirical survey was that an overwhelming 92% agreed that employability after completion of a degree was a parameter for measuring and for the institution's sustainability.
Using the data from the survey to further research this result, an analysis was conducted on the collected data on what respondents thought were the most important aspects of sustainability. Respondents were asked to rate the most important aspects of sustainability within an institution on a scale of 1-4, with 1 being "Not Important" and 4 being "Very Important". The subcategories of the STARS system were used as well as an additional subcategory of future employability. Figure 4.4 displays the results. After education, future employability is the most important aspect that respondents considered necessary for a university to be sustainable. This need is not currently covered in the STARS system, which may limit STARS's usefulness as a universal assessment system. § 4.6 Conclusions and Discussions § 4.

Interpretation of Results
A review of the lapses within the STARS system identified several major lapses which may limit it from being a universally useable assessment system. A major lapse can be considered anything that is not addressed within the assessment framework. The major lapses identified were the following: -The role of higher education institutions in building regional capacity to act in an increasingly competitive global economy, -Lack of post-graduate metrics, and -Lack of inclusion of employability after graduation.
Each of these major lapses is multi-faceted and needs to be addressed individually. However, there is a common trait that each of lapses share, which is the exclusion of post-graduation economic factors. More specifically, the metric of post-graduation employment is a critical exclusion that would partially address each of the major lapses. Employability, as defined by York (2004), is "a set of achievements -skills, understandings and personal attributes -that make graduates more likely to gain employment and be successful in their chosen occupations, which benefits themselves, the workforce, the community and the economy." For example, if a graduate is employed regionally it could be inferred that the higher education institution is assisting in building the regional capacity to act in an increasingly competitive global economy since there are available jobs. It also addresses part of the question of "What do graduates do in the world?" (Orr 2000) which takes into account the extent to which graduates still engage in sustainabilityrelated employment or other activities. And finally, it comprehensively appeases a key stakeholder demand of employability after graduation.
Along with these major lapses, a minor lapse was acknowledged in the research: the lack of clear definition of the Innovation criterion with STARS. This minor lapse does not necessarily deter STARS from being used universally, but does limit the effectiveness of the tool. § 4.

Discussion of Method Used for Comparison
The analytical breakdown of the weaknesses within the STARS sustainability assessment method found in the research is an important step towards making the assessment tool into a more universally accepted tool. However, as with any research that is empirical in nature, the limitation of this research needs to be considered in order to utilize the results effectively.
The first concern is the basis of this research. The research is a progression of two other peer-reviewed publications that were empirical in nature. The fact that this research furthers the previous findings is a limiting factor since the initial data is empirical. It is recommended that any use of these findings takes into full account the limited scope of this research, the characteristics of the previous studies reviewed including reviewed works' limitation and potential bias.
Another concern is relative to the bias of the survey, which was also addressed in the first paper by Maragakis and Dobbelsteen (2013). A major marketing campaign for the survey was conducted during an AASHE conference, which caters to sustainability-aware stakeholders. In the initial publication by Maragakis and Dobbelsteen (2013), the AASHE participants were isolated in order to note the bias. This paper accepts this bias and discloses that it may have influenced the results. As such, it is suggested that the results in this paper should be considered empirical and should best represent sustainability-aware stakeholders in higher education rather than all stakeholders within higher education. In order to ascertain more representative results, it is suggested that a much larger sample that encompasses the entire gamma of stakeholders in higher education is conducted. § 4.

Recommendations
Again, the inclusion of employability does not fully address the major lapses within the STARS framework, but provides a critical metric that would help make STARS a more appropriate system and universally useable. This research is not aware of any existing resources that combine general employability with sustainability in higher education. A multitude of work, such as Grant (2009) and Kemp (2011) have addressed the concept of sustainability-aware graduates and sustainability employment but none have dealt with the general term of employability.
Outside the realm of sustainability in higher education, the employability of graduates can be considered a much scrutinized topic. There are a multitude of assessments that deal directly or indirectly with employability after graduating a higher education institution. These assessment vary in scope and nature internationally, however they usually care at least one parameter that includes employability after graduation as a metric for success.
It is recommended that further research be conducting in order to find an already existing economic assessment that tracks employability after graduation and integrate it into the STARS framework. Although the criteria for finding an ideal economic assessment would need to be researched, it can be stipulated based on the findings of this paper that the assessment would need to be international in nature, already utilized by institutions and students for decision making and address the key lapses of STARS.
It is also recommended that further definition is applied towards the research criterion of STARS. Innovation is a key driver both in the field of sustainability and in higher education institutions. Innovation should have a clearer communicated methodology to address stakeholder needs.