Sustainability in Software Engineering: A Systematic Literature Review Birgit Penzenstadler, Veronika Bauer Coral Calero Xavier Franch Technische Universität München, Germany Universidad de Castilla - La Mancha, Spain Universitat Politècnica de Catalunya, Spain {penzenst|bauerv}@in.tum.de Coral.Calero@uclm.es franch@essi.upc.edu Abstract—Background: Supporting sustainability in software engineering is becoming an active area of research. We want to contribute the first Systematic Literature Review(SLR) in this field to aid researchers who are motivated to contribute to that topic by providing a body of knowledge as starting point, because we know from own experience, this search can be tedious and time consuming. Aim: We aim to provide an overview of different aspects of sustainability in software engineering research with regard to research activity, investigated topics, identified limitations, proposed approaches, used methods, available studies, and considered domains. Method: The applied method is a SLR in five reliable and commonly-used databases according to the (quasi-standard) protocol by Kitchenham et al. [1]. We assessed the 100 first results of each database ordered by relevance with respect to the search query. Results: Of 500 classified publications, we regard 96 as relevant for our research questions. We sketch a taxonomy of their topics and domains, and provide lists of used methods and proposed approaches. Most of the excluded publications were ruled out because of an unfitting usage of terms within the search query. Conclusions: Currently, there is little research coverage on the different aspects of sustainability in software engineering while other disciplines are already more active. Future work includes extending the study by reviewing a higher number of publications, including dedicated journal and workshop searches, and snowballing. I. M OTIVATION AND BACKGROUND Sustainability is currently an omni-present term in calls for research proposals and conference sessions (ICSE, CAiSE, RE, etc.). However, in literature, there is no overview of the current state of the art in supporting sustainability in software engineering research and practice. Consequently, researchers who are motivated to contribute to that topic (like the first author [2]) have to invest much time in finding a basic body of knowledge through literature research of many unrelated leads. This paper reports on our systematic literature review with the objective of retrieving a solid basis of knowledge1 on the support of sustainability in software engineering. The full protocol is available online as technical report [4]. 1 One of the common motivators for SLRs named by Zhang and Babar in [3, Tab.I]. A. Definition of Sustainability To clarify our research objective, we define our understanding of sustainability and what we mean by sustainability and how we want to apply it to software engineering. The most cited definition of sustainable development [5] is to “meet the needs of the present without compromising the ability of future generations to satisfy their own needs.” According to [5], sustainable development needs to satisfy the requirements of the three dimensions of society, economy, and environment. A fourth dimension, human sustainability, is less present in the public discussion. According to [6], it should be included, as it is the basis for the other dimensions. All four dimensions of sustainability are further detailed on in our SLR protocol [4]. B. Sustainability Aspects in Software Engineering Sustainability aspects can be brought to bear both during the development and use of software systems. We distinguish four aspects of sustainability in SE (orthogonal to the dimensions introduced in Sec. I-A). The development process viewpoint includes: • Development process aspect: Sustainability in the initial system development process (with responsible use of ecological, human, and financial resources). This aspect focusses on the initial conceptual and constructional development and we distinguish it from the late phase of actual system production for reasons of analysis. • Maintenance process aspect: Sustainability of the software system during its maintenance period until replacement by a new system (with continuous monitoring of quality, knowledge management). The product viewpoint encompasses the aspects of sustainability during production and usage: • System production aspect: Sustainability of the software system as product with respect to its use of resources for production (using green IT principles and sustainably produced hardware components). The actual system production happens after most of the initial development process and considers, inter alia, mass production aspects, logistics and factory organization issues. System usage aspect: Sustainability in the usage processes in the application domain triggered by the software system as product (responsible in impact on environment, using green business processes). We expect these aspects to have different scales of impact, growing from small to large in the order presented above, so that the system usage aspect potentially has the biggest impact (and, therefore, improvement potential). However, this is also dependent of the system under analysis. For our SLR, we are looking for all four aspects of sustainability in software engineering. The aspects imply different levels of abstraction and varied granularity, but nevertheless we are interested in the state of research for each of them. • Mahaux et al. [7] performed a preliminary search on the DBLP Computer Science Bibliography database3 . For articles with the prefixes “sustainab-” OR “ecolog-” OR “environmental-” in the title, the data base returned over 3000 results (in January 2010), but filtering on important software-related venues lead to as few as 11 results. They propose that a systematic literature review should be conducted. In contrast to [7], we are interested in publications from all scientifically sound venues and journals as we see great potential for learning from other domains. Therefore, we did not restrict this systematic literature review to softwarerelated venues, which is the main reason why we received more results. II. S EARCH D ESIGN AND P ROCESS C. A Body of Knowledge for Sustainability in SE Our research aim for the next years is to support the development of ICT systems for environmental sustainability (ICT4ES) with an adequate software engineering approach that integrates the knowledge of related disciplines that are concerned with sustainability. For that we need to build up on existing knowledge is SE as well as disciplines that have been related closer to sustainability, for example, environmental informatics. This research aim requires accumulating a body of knowledge for various reasons: justifying the basis for future research, learning as much as possible from other domains related to the topic, and providing a basis for other researchers as well as students who consider learning about and contributing to this area. One commonly accepted research method for accumulating a body of knowledge is a study in form of a systematic literature review [3]. D. Research Questions The overall research objective of the study is to find out what the current state of the art in supporting sustainability in software engineering research and practice is. This is further detailed in the following research questions: RQ1 How much activity was there in the last 20 years?2 RQ2 What research topics are being addressed? RQ3 What are the limitations of current research? RQ4 How is sustainability support performed? RQ5 Which methods are in use? RQ6 Are there case studies available? RQ7 Which domains are already considered? E. Related Work There are systematic literature reviews on different topics in software engineering, but so far none has been conducted that investigates the relation between sustainability and software engineering. 2 Our hypothesis is that most publications will be much younger, so a time span of 20 years ensures that we include all relevant ones. The search design and procedure follow the guidelines in [1]. As SLR research questions we directly adopted those enumerated in Sec. I-D. The search process for this study is based on an automated search of the following digital libraries: • IEEE Digital Library http://ieeexplore.ieee.org • ACM Digital Library http://dl.acm.org • SpringerLink www.springerlink.com • ScienceDirect / Scopus http://www.sciencedirect.com • Web of science http://apps.webofknowledge.com/WOS A. Search String The aim for our search string is to capture all results that relate sustainability or environmental issues with software engineering or requirements for software systems. The reasons for searching for requirements is that in this early development phase sustainability issues should emerge. The search string used on all databases is: (sustainab* OR environment* OR ecolog* OR green) AND (software engineering OR requirement OR software system) Although we explicitly list keywords in our search string that rather point to environmental sustainability, we expect to find all dimensions of sustainability. B. Inclusion Criteria We chose the following inclusion criteria in order to select the relevant publications to answer our research questions: • Publication date between 1/1/1991 - 31/12/2011 • Requirements phase of software development process • Explicit mentioning of software engineering • Scientific soundness • Relevance with respect to research questions • Analysis of sustainability-relevant application domains • Coverage of a SW ecosystem or SW sustainability 3 http://dblp.mpi-inf.mpg.de/dblp-mirror/index.php C. Exclusion Criteria • • “Environment” used in the sense of system environment, not nature. “Ecosystem” used as population of interacting systems, for example, agents. D. Roles and Responsibilities • • • • • Birgit Penzenstadler (TUM, principal researcher): IEEEXplore, result classification, detailed analysis Zolboo Ochirsukh, Elena Mircheva, Duc Tien Vu, Tuan Duc Nguyen (TUM, student research assistants): search on ACM, Web of Science, ScienceDirect, SpringerLink Veronika Bauer (TUM, expert reviewer): assessment of search result classification, review of detailed analysis Coral Calero (UCLM, expert reviewer): assessment of search result classification and detailed analysis Xavier Franch (UPC, expert reviewer): review of detailed analysis E. Article Selection Process The process was conducted as follows: 1) The researchers execute the search on each database and save the references in bibliography files. 2) The principal researcher reads all titles and abstracts and checks the inclusion and exclusion criteria for each entry. Major criterion is the topic of the content. 3) The principal researcher classifies the papers and articles according to type, topic, and domain. 4) The expert reviewers reassess the classification and inclusion/exclusion of search results. After their reassessment, we introduce an additional result classification: domain-specific papers that are interesting to learn from but not focussed on software engineering. 5) The principal researcher extracts statistics and analyses the included results in further detail. This is followed by a second assessment from the expert reviewers. F. Data Analysis The data is tabulated to show: • The databases and numbers of query results. (RQ1) • Listed by database for included publications: – Author, reference, date (RQ1) – Publication type and type of content (RQ6) – Topic of content (RQ2, RQ4, RQ5) – Application domain (RQ7) – Benefit for our body of knowledge (RQ4) • The number of relevant publications per year. (RQ1) • The respective venues and journals. (RQ1) Furthermore, the findings for RQ3, RQ4 and RQ5 are reported on separately. Due to the limitation of space, we provide the full list of references of the primary study as online appendix [8]. III. R ESULTS The overall number of results for each data base is listed in Tab. I. Table I N UMBER OF R ESULTS PER DATABASE Database IEEE Xplore ACM Digital Library SpringerLink ScienceDirect / Scopus Web of Science Date 27/12/11 26/12/11 29/12/11 29/12/11 29/12/11 Results 319.601 104.217 500.004 10.749 80.503 All results were ordered “by relevance” as displayed by the databases. From these results, we considered the first 100 results of each data base in our first iteration of the study. In total, we reviewed 500 publications. The following abbreviations are used to categorize the results in Tab. II-VI: • Publication: Kind of publication, e.g., Journal Article (A), Conference Paper (CP), Workshop Paper (WP), Book Chapter (BC), Letter to the editor (L) • Type: Kind of content presented in the publication, e.g., method, experience report, empirical study, tool • Topic: Short hint on principal content and keywords of the paper or article • Domain: Application or technology domain considered in the publication, e.g. transport, aviation, embedded systems, information systems, human aspects • Benefit: Classification of why we consider this publication to be relevant with respect to the research questions: Sustainability in software engineering (S in SE), sustainability-related application domains (S App Dom), sustainability (modeling) concept (S Concept), sustainable software solutions (S SW Sol), sustainable hardware solutions (S HW Sol) RQ1: How much activity was there in the last 20 years? We summarized the number of relevant publications per database in Tab. VII, per year in Tab. VIII, and per publication type in Tab. IX. In the last two years, there was a significant increase in the number of publications, and there was no publication included that was older than 2005, so our hypothesis for RQ1 holds. None of the results we included are older than 2005, but we did have older search results in the query evaluations, so this is not due to restricted availability online. Although we executed the search queries in late December, we already found journal articles dated to January 2012 in the results which we included as they were fully available. While the venues were relatively distributed, there was an accumulation of publications from “Environmental Modeling & Software” as well as the “Journal of Cleaner Production”. The fact that we classified many of the publications as “software solutions” or “sustainability-related application Table II I NCLUDED R ESULTS FROM IEEE X PLORE , FULL R EFERENCES IN A PPENDIX [8] Author and reference Date Pub. Type Topic Domain Benefit Kung et al. Middendorf et al. Ulieru Albertao et al. Zhou et al. 2011 2009 2010 2010 2008 CP CP CP CP CP method method method metrics method green decision-making framework envir. aspect in mechatronics design design for resilience of networked critical infrastructures sustainability performance of software green remanufacturing engineering in structural machinery building construction mechatronics, robots digital ecosystems eBusiness security tech. S S S S S App Dom App Dom App Dom Concept App Dom Table III I NCLUDED R ESULTS FROM ACM, FULL R EFERENCES IN A PPENDIX [8] Author and reference Date Pub. Type Topic Domain Benefit Mouton et al. Geist et al. O’Sullivan Penzenstadler et al. Zhongjia et al. Audisio et al. Miginsky et al. Pennington et al. Blevis et al. Adomavicius et al. Pereira et al. Henriksen et al. Dick et al. Ticehurst et al. Shih et al. Cushing et al. Cushing et al. Kang et al. Amsel et al. Choucri et al. Vicat-Blanc et al. DesAutels et al. Ramona Mathevet et al. Salski Abidin et al. Kase et al. Prabhakar et al. Umstatter Vance Pousman et al. Fu et al. 2009 2009 2010 2011 2010 2009 2008 2008 2007 2007 2009 2007 2011 2007 2010 2007 2006 2008 2010 2011 2011 2011 2009 2007 2007 2010 2008 2010 2011 2007 2008 2011 A A CP CP CP A A A CP A A A CP A A A CP A CP WP BC A CP A CP A CP A A CP CP CP model challenges method method prototype method method method reflection model method method model model model method overview method tool model model model method tool method tool model model review model method framework habitat suitability models for river management computer science challenges at exascale data mining for biodiversity prediction in forests teach sustainability in software engineering design of self-propelled walking brush cutter hazard management in two Alpine river basins computer reconstruction of the biological networks indirectly driven knowledge modeling in ecology design critique as research to link sustainability and interactive technologies technology roles & influence in an ecosystem model of technology evol. knowledge discovery for coastal waters classification public participation modeling in management of groundwater contamination meta-design environments to motivate changes in energy consumption assessing the sustainability of coastal lakes butterfly and wetland ecology for context-aware ubiquitous learning database design for ecologists eco-informatics and natural resource management GIS-based poultry litter management system for nutrient planning tool for estimating the energy consumption of software simulation modeling approach to evaluate renewable energy readiness optical networks and cloud as architecture for a sust. future internet explore the market price of “sustainable” notebooks consolidating eco-economics through financial and fiscal instruments role-playing game for collective awareness of wise reedbed use fuzzy approach to ecological data analysis tool to detect and predict urban growth pattern sustainable informal it learning in community-based nonprofits transfer scheme for energy harvested WSN gateways review of the evolution of virtual fences permanent coexistence for a linear response omnivory model casual information visualization of printer data urban media framework of social innovation and service design ecological knowledge high performance computing forestry, data mining education digital manufacturing GIS tool ecological networks knowledge models design research technology ecosystem environmental monitoring groundwater management energy sustainability environmental management mobile learning ecosystem info management eco-informatics decision support green computing energy simulation cloud life cycle analysis eco-culture environmental education fuzzy systems neural network education energy grids electronics in agriculture modeling sustainability design service design S App Dom S SW Sol S App Dom S in SE S HW Sol S App Dom S SW Sol S Concept S Concept S Concept S SW Sol s conc S SW Sol S Concept S App Dom S SW Sol S Concept S App Dom S SW Sol S App Dom S App Dom S App Dom S Concept S App Dom S Concept S App Dom S Concept S App Dom S App Dom S App Dom S SW Sol S Concept Table IV I NCLUDED R ESULTS FROM S CIENCE D IRECT, FULL R EFERENCES IN A PPENDIX [8] Author and reference Date Pub. Type Topic Domain Benefit Abdulaziz et al. Alain et al. Alexandrov Ashraf et al. Beusen et al. Bovea et al. Brown et al. Cardona et al. Cui et al. Faith-Ell et al. Fan et al. Fuller et al. Harmon et al. Hughes et al. Justyna et al. Kalivarapu et al. Kit et al. Koormann et al. Kubba Liu et al. Mei et al. Naumann et al. Rizzo et al. Smith et al. Tong et al. Tseng et al. Xu et al. Yen et al. Zhang et al. Zhang et al. 2011 2006 2011 2012 2011 2012 2010 2011 2009 2006 2007 2006 2009 2010 2010 2008 2012 2006 2010 2011 2010 2011 2006 2010 2011 2010 2010 2011 2011 2010 A A L A A A A A A A A A A A A A A A BC A A A A CP A A A A A A review method assessment method tool tool review tool model method case study model tool evaluation framework review framework tool model strategy study review model evaluation method method study review study study model car parking requirements for sustainable transport development modeling living systems, their diversity and their complexity technical assessment and evaluation of environmental models image data fusion for the remote sensing of freshwater environments dynamic simulation and visualization software for mathematical models taxonomy of ecodesign tools for integrating environmental requirements software tool designed to verify ensemble forecasts of numeric variables software package developed for dynamic simulation of water quality in rivers management-oriented valuation for ecol. water requirements for wetlands application of environmental requirements in Swedish road maintenance contracts model for China’s energy requirements and CO2 emissions analysis software package for optimizing connectivity in conservation planning exploratory evaluation of the market case for green energy determination of environmental water requirements for rivers green roof performance towards management of runoff water quantity and quality software framework for modeling of contaminant transport in groundwater texture-based identification of urban slums in India using sensing data modeling down-the-drain chemicals in rivers green project requirements and strategies energy requirements and carbon dioxide emissions of tourism industry research progress of ecological water requirement in china reference model for green and sustainable software and its engineering dynamic systems-based software packages for ecological systems green product design through product modularization using atomic theory generating the plan of mandatory green space in urban systems evaluating a firm’s green supply chain management review on ecological engineering based engineering management management’s role in adopting green purchasing standards in industry costs and barriers of green property development in China multi-source remote sensing data for estimating ecological water requirement transport agronomy systems environmental modeling applied geography environmental modeling clean production environmental modeling environmental modeling nature conservation cleaner production energy biodiversity protection green energy environmental modeling ecological engineering environmental modeling applied geography environmental modeling architecture energy ecological informatics software engineering environmental modeling lean manufacturing urban development cleaner production management business research property development environmental modeling S S S S S S S S S S S S S S S S S S S S S S S S S S S S S S App Dom App Dom Concept App Dom SW Sol in SE SW Sol SW Sol App Dom App Dom App Dom SW Sol App Dom App Dom App Dom SW Sol SW Sol Concept Concept App Dom App Dom in SE SW Sol Concept App Dom App Dom Concept Concept App Dom SW Sol Table V I NCLUDED R ESULTS FROM S PRINGER L INK , FULL R EFERENCES IN A PPENDIX [8] Author and reference Date Pub. Type Topic Domain Benefit Huang et al. Liao et al. Liu et al. Du May et al. 2009 2009 2008 2010 2006 CP CP A A CP study method method method method support green customers’ decision process on electronic commerce multimedia stream format and green design concept for e-learning wheat growth model multi-agent system neural network control for greenhouse management composing biological workflows through web services web engineering education computing in agriculture computing in agriculture parallel processing S S S S S App Dom Concept SW Sol SW Sol SW Sol Table VI I NCLUDED R ESULTS FROM W EB O F S CIENCE , FULL R EFERENCES IN A PPENDIX [8] Author and reference Date Pub. Type Topic Domain Benefit Alvarez et al. Balana et al. Boose et al. Bravi et al. Ellison et al. Greene et al. Hall et al. I-Wah Jia et al. Jin et al. Kaduk et al. Lundy et al. McCabe McIntosh et al. Scheller et al. Seppala et al. Singh et al. Tao et al. Verweij et al. Wang et al. Wang et al. Xu et al. Zhang et al. Zhou et al. 2011 2011 2007 2011 2006 2010 2011 2011 2011 2011 2011 2011 2006 2007 2010 2011 2011 2008 2010 2008 2009 2007 2011 2008 A A CP A A A A A A A A A A A A A A CP A CP CP A A CP method review method assessment model method method analysis case study study method method overview method method assessment method model perspective method tool evaluation assessment method decision making for treatment intensity in purifying plants cost-effectiveness analysis of agri-environmental measures reliable datasets for environmental models with an analytic web life cycle assessment of a micromorph photovoltaic system analytic webs support the synthesis of ecological data sets decision analysis with exploration and evaluation phases requirements for 3D vegetation structure from space development and conditions of home-school cooperation urban wetland planning in Beijing integrated calculation of ecological water demand for basin systems redicting the time of green up in temperate and boreal biomes integrating sciences to sustain urban ecosystem services sustainable building design in Australia database design for ecologists including observation data increasing the reliability of ecological models using SE techniques greenhouse gas emissions and material flows in Finland resource conservation technology in rice-wheat cropping system UML-based green alignment selection decision making model IT perspective on integrated environmental modeling systematic research on the cost control of the green industry web-based distributed certification system of green food sustainability evaluation of a nature reserve project combined biostabilization and landfill for solid waste green remanufacturing engineering in structural machinery wastewater water pollution ecological data sets energy ecological data sets land management environmental modeling education ecological complexity water demand climate change geography eco-architecture ecoinformatics software engineering clean production environment intelligent computation software engineering cost control env. science environmental management environmental management security technology S S S S S S S S S S S S S S S S S S S S S S S S domain”, some more as “sustainability concepts” and only few as “sustainability in software engineering” implies that there is still relatively little research published that could be considered for building up a body of knowledge. Table VII I NCLUDED R ESULTS PER DATABASE Name IEEEXplore ACM Digital Library Springer Link Science Direct Web of Science Total Number of Included Results 5 out of 100 32 out of 100 5 out of 100 30 out of 100 24 out of 100 96 out of 500 Table VIII I NCLUDED R ESULTS PER Y EAR Year 1991 - 2005 2006 2007 2008 2009 2010 2011 2012 Total Number of Results 0 9 12 11 11 21 29 3 96 Table IX I NCLUDED R ESULTS PER P UBLICATION T YPE Publication Type Journal articles Book chapters Conference papers Workshop papers Letters to the editor Technical reports Total Number of Results 65 2 27 1 1 0 96 App Dom App Dom SW Sol App Dom SW Sol App Dom SW Sol Concept App Dom App Dom App Dom Concept App Dom SW Sol in SE App Dom App Dom in SE in SE Concept SW Sol App Dom App Dom App Dom RQ2: What research topics are being addressed? For a quick illustrated overview, we have generated a weighted topic cloud from keywords, taken from the titles and abstracts, that visualizes the topics in Fig. 1. We have derived a taxonomy for the addressed research topics in Fig. 2 that abstracts from some of the details listed in the original classifications tables in Tab. II-VI. The dimensions of the taxonomy are the degree of domain specifity, from general purpose to domain-specific research and the indexing between analytical approaches (frameworks and assessment) and constructive approaches (methods and tools). The taxonomy shows a tendency towards domain-specific, constructive approaches. There are not many publications rated as general purpose, and there is little methodical guidance for supporting sustainability. Both the keyword cloud and taxonomy rely strictly on keywords taken from titles and abstracts. Nevertheless, their reproduction might reveal slightly varied results, but we do not consider that a problem as we use them only to give an overview of topics without deriving any further statistics from them. RQ3: What are the limitations of current research? To identify limitations of current research, we reviewed our classification of topics and application domains in Tab. II-VI. We performed a pragmatic and informal gap analysis that resulted in three major limitations: • High complexity. Reason: Due to the high connectivity between the different aspects of sustainability, (software) systems engineering becomes highly complex. This is visible in knowledge management approaches, e.g., [9] and decision support systems, e.g., [10]. Conclusion: High complexity requires clear concept definitions and consistent, traceable models. One method to cope with high system complexity that might prove helpful is systems’ thinking [11]. Figure 1. Weighted topic cloud, created with http://www.wordle.net/ ecological engineering analytical (frameworks) green decision-making framework green purchasing standards in industry evaluation of environmental models green property development in China agri-environmental measures framework of social innovation evaluation of a nature reserve project cost control of the green industry green customersʼ decision on electronic commerce market case for green energy assessing the sustainability of coastal lakes green supply chain management sustainable building design in Australia price of sustainable notebooks urban ecosystem services technology roles and paths of influence knowledge modeling in ecology environmental aspect in mechatronics valuation method for ecological water requirements micromorph photovoltaic system architecture for a sustainable future internet green and sustainable software and its engineering green space in urban systems meta-design environments to motivate changes in energy consumption constructive (methods) treatment intensity in purifying plants environmental water requirements for rivers data mining for biodiversity prediction requirements in Swedish road maintenance decision analysis with exploration phase sustainability performance of software Chinaʼs energy requirements habitat suitability models certification system of green food energy requirements of tourism industry eco-informatics and natural resource management wetland planning in Beijing simulation modeling approach to evaluate renewable energy green roof performance modeling living systems teach sustainability modeling in management of groundwater contamination environmental modeling greenhouse gas emissions and material flows in Finland green alignment selection decision making model green design concept for e-learning linear response omnivory model product modularization using atomic theory green remanufacturing engineering butterfly and wetland ecology for context-aware ubiquitous learning 3D vegetation structure from space sustainable transport development database design for ecologists synthesis of ecological data sets down-the-drain chemicals in rivers network control for greenhouse management hazard management biostabilization and landfill for solid waste fuzzy approach to ecological data analysis poultry litter management system for nutrient management energy harvested WSN gateways conditions of home-school cooperation design for resilience of infrastructures green up in boreal biomes it learning in community-based nonprofits predict urban growth pattern awareness of wise reedbed use environmental models with an analytic web remote sensing of freshwater environments reconstruction of biological networks resource conservation technology in rice-wheat cropping composing biological workflows modeling of contaminant transport in groundwater wheat growth model multi-agent system visualization software for mathematical models dynamic simulation of water quality urban slums in India using sensing data tool for estimating the energy consumption software packages for ecological systems estimating ecological water requirement visualization of printer data sustainability and interactive technologies ecodesign tools ecological water demand for basin systems verify ensemble forecasts of numeric variables optimizing connectivity in conservation planning general purpose • design of brush cutter domain-specific Figure 2. • knowledge discovery for coastal waters evolution of virtual fences Taxonomy of research topics High domain-specifity. Reason: The frameworks and methods we found within the results are highly domain-specific, e.g., [12], [13]. This is also visible in the higher density of domainspecific approaches in Fig. 2. Conclusion: Effective approaches for supporting sustainability require specific domain knowledge. Software engineering. Reason: There is only one approach in software engineering that explicitly addresses sustainability. It is a reference framework with specific application in web engineering [14]. Conclusion: An encompassing reference framework for SE is still missing. RQ4: How is sustainability support performed? Constructive support for sustainability is performed by frameworks, models, methods, and metrics (Tab. X). Thereby, most approaches are specific to a special application domain, as visible by the density on the domain-specific side in Fig. 2. • Frameworks, e.g., for civil engineering [12] or contaminant transport [15] Table X I NCLUDED R ESULTS PER C ONTENT T YPE Class Constructive Empirical Discussion Type of Content Method Model Metrics Framework Tool/Prototype Review Study Evaluation Assessment Overview Challenges Analysis Reflection Perspective Strategy Number of Results 36 18 1 2 9 6 8 3 4 2 1 1 1 1 1 We have derived a taxonomy for the domains that were used and described in the publications in Fig. 4. We used the same dimensions as for the research topics taxonomy in Fig. 2 and identified five coarse-grained domain clusters: Systems & Knowledge in the area of general purpose, analytical approaches, Technologies & Methods on the constructive side of the general purpose dimension, Education somewhere in the middle between these two, special Disciplines provide more domain-specific, analytical approaches, and the corresponding Application & Implementation cluster contributes the domain-specific, constructive approaches. These clusters are not overlap-free, but only a means to illustratively structure their diversity. The terms within the cluster clouds in Fig. 4 indicate the individual domains. IV. D ISCUSSION Models, e.g., for software systems [14] or databases [16] • Methods for specific application areas, e.g., security technology [13], green product design [17], or ecology knowledge [9] • Metrics, e.g., for sustainability in eBusiness [18] We chose just a few of the approaches for illustration and preferred the ones that are rather close to sustainability in software engineering. Furthermore, there are some empirical publications and rather few discussions. • RQ5: Which methods are in use? There is a wide variety of methods in use for different purposes — we found traditional software engineering techniques as well as domain-specific techniques and methods from other disciplines. Many approaches apply entity-relationship modeling, e.g. [9], as means to represent their data, knowledge, or information models. Neural networks are in use for dynamic environments and simulations, e.g. [19]. Methods adapted from other disciplines are, inter alia, cost calculations, e.g. [20], and life cycle analysis, e.g. [21]. RQ6: Are there case studies available? We classified publications as case studies when they were explicitly named as such in the abstract and they were not, for example, only containing a small illustrative case study within a method proposal. The studies are listed in Tab. XI. Unfortunately, none of the studies contributes explicitly to an understanding of how to develop software for sustainable systems, but rather to analyses of specific application domains. Furthermore, publications that promote studies are often method proposals illustrated in a case study performed by the principal researcher. RQ7: Which domains are already considered? For an illustrated overview, please see the weighted domain cloud that visualizes the application domains in Fig. 3. This section provides a discussion of the results and of the threats to validity for this study. A. Conclusions on the State of the Art We started our search expecting to find more results to be classified as Sustainability in Software Engineering (S in SE in column Benefit in Tab. II-VI). As we found less than expected for a body of knowledge on S in SE, we decided to extend the inclusion to publications that we classified as a research we could learn from when further investigating sustainability in software engineering. This lead to the other Benefit categories S Concept, S App Dom, S SW Sol, and S HW Sol as explained in Sec. III. In our opinion, there is still a lot of research work to be done, especially to support the different dimensions of sustainability from within the software engineering discipline. This can either occur in form of domain-independent guidelines or domain-specific methods. B. Conclusions for a Body of Knowledge Due to these findings, our envisioned Body of Knowledge has areas that represent the core S in SE publications, plus areas that represent application domains with software and hardware solutions as well as sustainability concepts from related disciplines that we can learn from. This is illustrated in in Fig. 5. Body Of Knowledge S Concept (22 results) Figure 5. S SW Sol (25 results) S in SE (9 results) S App Dom (47 results) S HW Sol (1 result) Areas of the Body of Knowledge for S in SE Table XI C ASE S TUDIES Author and ref. Huang et al. [10] Faith-Ell et al. [22] Liu et al. [23] Tseng et al. [24] Yen et al. [25] Zhang et al. [26], [27] Jia et al. [28] Jin et al. [29] Domain web engineering cleaner production energy cleaner production business research property development ecology hydrology Context support green customers’ decision process on electronic commerce application of environmental requirements in Swedish road maintenance energy requirements and carbon dioxide emissions of tourism industry evaluating a firm’s green supply chain management management’s role in adopting green purchasing standards in industry costs and barriers of green property development in China urban wetland planning in Beijing ecological water demand for basin systems Figure 3. Applied method questionnaire and experiment semi-structured interviews index decomposition analysis relational analysis, experiment questionnaires cost analysis ecological complexity research integrated calculation Weighted domain cloud, created with http://www.wordle.net/ analytical (frameworks) Systems / Knowledge service design life cycle analysis digital ecosystems technology ecosystem ecological knowledge Disciplines ecosystem information management mechatronics business research knowledge models production management ecological engineering electronics geography clean production high performance computing Education design research security e-learning biodiversity protection agronomy architecture web engineering informatics software engineering nature conservation sustainability design environmental education eco-culture Application / Implementation eBusiness eco-informatics data mining mobile learning constructive (methods) GIS agriculture environmental monitoring parallel processing cloud building construction forestry property development Technologies / Methods neural network transport urban development decision support energy smart grids groundwater management environmental management digital manufacturing ecological networks fuzzy systems robots lean manufacturing green computing energy simulation general purpose domain-specific Figure 4. Taxonomy of application domains C. Threats to Validity There is a number of threats to validity that we are aware of and tried to minimize by different mitigation actions. 1) Researcher’s bias: The semi-automatic part of the search was performed by five researchers. There could be a researcher’s bias as the first selection was performed by only one researcher. We minimized the effects of such a bias by two measures: • We explicitly stated the research questions, inclusion and exclusion criteria, and the rationale for performing the search. • The first selection was reviewed and assessed by two expert reviewers from different institutions (TUM, UCLM). Differences were subsequently discussed, resolved and commonly agreed upon. • The detailed analysis of the principle researcher was reviewed by all three expert reviewers (TUM, UCLM, UPC). 2) Search string validity: The search string validity can be questioned under two aspects: On one hand whether it filtered out too many publications that would have been relevant, and on the other hand whether it included too many irrelevant results and was, in either case, not the adequate search string. Indicators for too many false positives are purely hardware papers, but as the automatically found Green IT publications all contained part of the second parenthesis of the search string, they were included in the results. Then, purely application in environmental domains, for example, agricultural support systems with no explicit relation to sustainability but relevant in case they exhibited an explicit link to sustainability in their content. Furthermore, “environment” used in the sense of system environment, not nature — these samples had to be excluded by hand as well as “ecosystem” used as population of interacting systems, for example, agents. Indicators for too many relevant exclusions were that we found less relevant results than we would have expected. This can either be due to a search string that was too restrictive, to a search that was not extensive enough, or to the fact that there is rather little published yet on that specific topic. Not all publications we would have expected showed up early in the search results. For example, we missed Cabot et al. [30], as they treat goal modeling for supporting sustainability in the context of conference organisation. Mahaux et al. [7] were also missing in the results, with their work on exploring sustainability requirements.4 3) Database query evaluation: We did not have any information on which database performed which kind of search query evaluation, and a lazy versus an eager database query evaluation of the search string would probably have a 4 These works were not included into the results manually because we wanted to strictly follow the SLR method. However, they will be included in the extended version and the envisioned body of knowledge. significant impact on the search results, considering that we reviewed the first 100 most relevant results. In case of a “lazy” search string evaluation, the results might have included more references matching early parts of the search string than compared to matching later parts. In that case, the results might be slightly biased in terms of favoring the terms “sustainab*” and “software engineering” and subordinating “green” and “software systems”. As many of the results contained the term “software system” and not “software engineering”, we are confident that there was no bias introduced by database query evaluation. 4) Cross-validation of the search engines: We received hardly any double entries in the automatic search results. We would have expected some double entries in the more general databases ScienceDirect and WebOfScience. We decided not to use meta search engines in our first iteration of the SLR because relying on only one meta search engine would have made us completely dependent of the reliability of that engine, and using various meta search engines would have led to highly redundant results, as a pre-check showed. Interestingly, Web of Science found Estrin [31] highly ranked, which originates from IEEEXplore but was not included in the IEEEXplore results (at least not within the first 100 results). This might be a hint towards different search query evaluation. It would be one interesting step in future work to replicate the searches on more databases and meta search engines and explicitly compare the coverage. V. C ONCLUSIONS In this paper, we presented the results of our SLR [4] on the research activity in sustainability in software engineering and related topics that allow for building up a body of knowledge. We considered 96 of 500 reviewed publications relevant with respect to our research questions and classified them according to content, topic, application domain, and potential benefit for further investigation. On that basis, we provided taxonomies for represented research topics and application domains. As there were not as many publications explicitly presenting work on sustainability in software engineering than expected, we propose an extended body of knowledge for S in SE that includes related application domains and sustainability concepts from related disciplines that we can learn from when further investigating S in SE. Future work is to extend the study in two directions: on one hand by snowballing (following references) and on the other hand via meta search engines, book search engines, and dedicated journal searches. Probably even more important is the challenge of making SLRs themselves “sustainable” by providing yearly updates that not only repeat an SLR but adapt the iterations over the years according to lessons learned from previous iterations. Thereby, we can establish stable bodies of knowledge. R EFERENCES [1] B. Kitchenham and S. Charters, “Guidelines for performing systematic literature reviews in software engineering,” tech. rep., Software Engineering Group, Keele University, 2007. [2] B. Penzenstadler, B. Tomlinson, and D. Richardson, “Support environmental sustainability by requirements engineering,” in International Workshop on Requirements Engineering for Sustainable Systems, 2012. [3] H. Zhang and M. Babar, “An empirical investigation of systematic reviews in software engineering,” in Empirical Software Engineering and Measurement (ESEM), 2011 International Symposium on, pp. 87 –96, sept. 2011. [4] B. Penzenstadler, V. Bauer, and C. Calero, “Sustainability in software engineering: A systematic literature review for building up a knowledge base,” tech. rep., Technische Universität München, 2012. [5] United Nations World Commission on Environment and Development, “Report of the World Commission on Environment and Development: Our Common Future,” in United Nations Conference on Environment and Development, 1987. [6] R. Goodland, Encyclopedia of Global Environmental Change, ch. Sustainability: Human, Social, Economic and Environmental. John Wiley and Sons, 2002. [7] M. Mahaux, P. Heymans, and G. Saval, “Discovering Sustainability Requirements: an Experience Report,” in 17th International Working Conference on Requirements Engineering: Foundation for Software Quality, 2011. [8] B. Penzenstadler, V. Bauer, C. Calero, and X. Franch, “Appendix: References of the Included Results of the SLR Sustainability in Software Engineering.” http://www4.in.tum. de/∼penzenst/sources/ease2012 slr appendix.pdf. [9] D. D. Pennington, I. N. Athanasiadis, S. Bowers, S. Krivov, J. Madin, M. Schildhauer, and F. Villa, “Indirectly driven knowledge modelling in ecology,” International Journal of Metadata, Semantics and Ontologies , Volume 3 Issue 3, 2008. [10] F.-H. Huang, Y.-L. Lee, and S.-L. Hwang, “E-shopping behavior and user-web interaction for developing a useful green website,” in Human-Computer Interaction. New Trends (J. Jacko, ed.), vol. 5610 of Lecture Notes in Computer Science, pp. 446–454, Springer Berlin / Heidelberg, 2009. Springer. [11] D. Meadows, Thinking in Systems: A Primer. Chelsea Green Publishing, 2008. [12] P. Kung, V. C. Chen, and A. Robinson, “Multivariate modeling for a multi-stage green building framework,” in 2011 IEEE International Symposium on Sustainable Systems and Technology (ISSST), pp. 1–6, IEEE, May 2011. [13] G. Zhou, G. Duan, H. Wu, and T. Yao, “Green remanufacturing engineering in structural machinery based on reverse engineering,” in International Conference on Security Technology, 2008. SECTECH ’08, pp. 221–225, IEEE, Dec. 2008. [14] S. Naumann, M. Dick, E. Kern, and T. Johann, “The greensoft model: A reference model for green and sustainable software and its engineering,” Sustainable Computing: Informatics and Systems, no. 0, pp. –, 2011. [15] V. Kalivarapu and E. Winer, “A multi-fidelity software framework for interactive modeling of advective and diffusive contaminant transport in groundwater,” Environmental Modelling & Software, vol. 23, no. 12, pp. 1370 – 1383, 2008. [16] J. B. Cushing, N. Nadkarni, M. Finch, A. Fiala, E. MurphyHill, L. Delcambre, and D. Maier, “Component-based enduser database design for ecologists,” Journal of Intelligent Information Systems , Volume 29 Issue 1, 2007. [17] S. Smith and C.-C. Yen, “Green product design through product modularization using atomic theory,” Robotics and Computer-Integrated Manufacturing, vol. 26, no. 6, pp. 790 – 798, 2010. 19th International Conference on Flexible Automation and Intelligent Manufacturing: Lean manufacturing and Services. [18] F. Albertao, J. Xiao, C. Tian, Y. Lu, K. Q. Zhang, and C. Liu, “Measuring the sustainability performance of software projects,” in 2010 IEEE 7th International Conference on eBusiness Engineering (ICEBE), pp. 369–373, IEEE, Nov. 2010. [19] L. Du, “Expert control based on neural networks for controlling greenhouse environment,” in Computer and Computing Technologies in Agriculture III (D. Li and C. Zhao, eds.), vol. 317 of IFIP Advances in Information and Communication Technology, pp. 126–132, Springer Boston, 2010. Springer. [20] C. Ramona, “Creating and consolidating eco-economics through financial and fiscal instruments. contribution of green taxes,” Proceedings of the 8th WSEAS international conference on System science and simulation in engineering, 2009. [21] M. Bravi, M. L. Parisi, E. Tiezzi, and R. Basosi, “Life cycle assessment of a micromorph photovoltaic system,” Energy, vol. 36, pp. 4297–4306, July 2011. [22] C. Faith-Ell, B. Balfors, and L. Folkeson, “The application of environmental requirements in swedish road maintenance contracts,” Journal of Cleaner Production, vol. 14, no. 2, pp. 163 – 171, 2006. [23] J. Liu, T. Feng, and X. Yang, “The energy requirements and carbon dioxide emissions of tourism industry of western china: A case of chengdu city,” Renewable and Sustainable Energy Reviews, vol. 15, no. 6, pp. 2887 – 2894, 2011. [24] M.-L. Tseng and A. S. Chiu, “Evaluating firm’s green supply chain management in linguistic preferences,” Journal of Cleaner Production, no. 0, pp. –, 2010. [25] Y.-X. Yen and S.-Y. Yen, “Top-management’s role in adopting green purchasing standards in high-tech industrial firms,” Journal of Business Research, no. 0, pp. –, 2011. [26] X. Zhang, A. Platten, and L. Shen, “Green property development practice in china: Costs and barriers,” Building and Environment, vol. 46, no. 11, pp. 2153 – 2160, 2011. [27] X. Zhang, L. Shen, and Y. Wu, “Green strategy for gaining competitive advantage in housing development: a china study,” Journal of Cleaner Production, vol. 19, no. 2-3, pp. 157 – 167, 2011. [28] H. F. Jia, H. T. Ma, and M. J. Wei, “Urban wetland planning: A case study in the beijing central region,” Ecological Complexity, vol. 8, pp. 213–221, June 2011. [29] X. Jin, D. H. Yan, H. Wang, C. Zhang, Y. Tang, G. Y. Yang, and L. H. Wang, “Study on integrated calculation of ecological water demand for basin system,” Science Chinatechnological Sciences, vol. 54, pp. 2638–2648, Oct. 2011. [30] J. Cabot, S. Easterbrook, J. Horkoff, L. Lessard, S. Liaskos, and J.-N. Mazon, “Integrating sustainability in decisionmaking processes: A modelling strategy,” in Software Engineering - Companion Volume, 2009. ICSE-Companion 2009. 31st International Conference on, pp. 207 –210, May 2009. [31] D. Estrin, “Participatory sensing: Applications and architecture,” Ieee Internet Computing, vol. 14, pp. 12–14, Jan. 2010.