KDE Eco 2021 Sprint: Details On The Community Planning For KDE's Ecological Project
This is a follow-up to the KDE Eco Sprint announcement posted on The Dot.
On 11 December 2021, KDE Eco held the first of many planned Sprints, with 7 people in attendance. The Sprint was originally intended to be an in-person event to set up a community measurement lab, but Corona had other ideas. Nevertheless, the community deployed its usual resourcefulness, and we met online instead.
KDE Eco 2021 Sprint Announcement On KDE News
In this long-form post I will provide a detailed overview of the main issues discussed at the Sprint and their outcomes; see also the minutes for further details.
Did you know?
Discussions similar to these occur monthly at our community meetups on the 2nd Wednesday of the month from 19h-20h CET/CEST (Berlin Time).
Creating a KDE Eco team
KDE Eco currently consists of three related projects: (i) Free and open source Energy Efficiency Project (FEEP), (ii) Blauer Engel For FOSS (BE4FOSS), and (iii) Blauer Engel Applications. Each project has its own particular focus: FEEP is concerned with measuring the energy consumption of Free Software, BE4FOSS is focused on community organization and spreading information about the Blauer Engel eco-label, and the Blauer Engel Applications repository hosts all documents related to certifying KDE/Free Software with the eco-label. Until the Sprint the three repositories for the projects were hosted at various personal namespaces. Now, all three repositories can be found at https://invent.kde.org/teams/eco.
Having the projects under one team more clearly links these and possible future projects. Moreover, having a non-personal namespace is more appropriate for community projects. Feel free to check out the activities of each project at the above team space!
Teams > KDE Eco
Completing the Blauer Engel application for Okular
There are three KDE applications whose energy consumption has already been measured: Okular, KMail, and Krita. Of these three, the Blauer Engel applications for Okular and KMail have been in preparation for some time, and one goal was to complete Okular's application. We are proud to announce that, following the Sprint, Okular's application has been submitted and is currently under review for certification. We expect feedback from RAL gGmbH, the awarding body for the Blauer Engel, within 2-3 months. KMail's submission will follow soon.
During a group review of the Okular application, participants raised many interesting points. One point, for instance, came up when discussing the transparency criteria of the Blauer Engel (Section 3.1.3.2). Malte Reißig from the IASS Research Group on "Digitalisation and Sustainability Transformations" highlighted how open development may positively affect some of the other requirements of the award criteria (see this discussion), such as the long-term maintainability and user autonomy of a software product. That is, Free Software users do not only passively use software, but they are encouraged and able to express wishes for the future development of applications, and may actively participate in the planning of milestones. Moreover, with open development, commit messages can be linked to specific issues or bugs reported by a community of users and developers. With Free Software, an application is not just an end product, but an open process reflecting the needs and wants of its users and their communities. With FOSS, the community drives the direction of development!
Okular Icon
Another point was related to the distribution of Free Software and how it presents challenges for the fulfillment of the Blauer Engel criteria. Since Free Software typically distributes software in a decentralized way, for any one application there are numerous release and distribution channels. Consider Okular, which is packaged not only for Flatpak, but also Microsoft Store, not to mention numerous GNU/Linux distros (e.g., OpenSuse, Debian). Differences in packaging infrastructure may affect the fulfillment of the Blauer Engel requirements, such as uninstallability, modularity, etc.
Companies releasing proprietary software, typically with centralized deployment of their products, will not have these issues. In light of this, one possibility the group discussed is certifying only the source code of a program, therefore remaining neutral about differences in how a software product is packaged and distributed. Another possibility is certifying distribution-specific releases, such as that for the Microsoft Store. In fact, the latter could be used for marketing purposes to reach new users and bring them into the KDE community -- see, e.g., this discussion.
We would love to know what you think. Please join the conversation on our mailing list or Matrix room or via the issue trackers at our GitLab repositories!
Standard Usage Scenarios
A Standard Usage Scenario (SUS) reflects the typical functions of an application, with the frequency of those functions taken into account. Standard Usage Scenarios are central to measuring the energy consumption of a piece of software.
When reviewing the SUS action logs for Okular for the Blauer Engel application, two important points were made that will improve implementation of SUS when measuring in the lab. One was the observation about the need to document not only actions but also the objects of actions (e.g., when using a pdf-reader, which pdf is opened), as this can influence the energy-consumption. The other was that including idle action is important when defining a SUS: idle action is both realistic and can show nothing is happening when nothing is supposed to be happening.
Several other issues were brought to the virtual table. Nicolas Fella, a software engineer at KDAB Berlin, expressed interest in comparing two chat clients, which shifted the conversation to defining single scenarios when comparing two applications with similar usage. A single SUS will limit which software functions can be tested, but it will make results from the measurements directly comparable. Moreover, for client-server software such as a chat or email clients, there is the issue of the network component, which for controlled testing would require setting up and measuring the energy consumption of a server. Former KDE e.V. president and long-time KDE contributor Cornelius Schumacher pointed out that for the KMail measurements only the computing work on the local machine was measured. Measuring non-local computing in client-server software is more relevant than ever in today's world, and this is currently planned for the revised Blauer Engel award criteria for software.
KMail Welcome Screen
Another topic discussed was defining SUS actions abstractly in order to automate the implementation of usage scenarios with different tools (Actiona, xdotool, KXmlGui, etc.). An important point was that different emulation tools do different things, which could present challenges to automating the process. For instance, one cannot type text when using KXmlGui for emulation, and thus something like xdotool may still be needed. On a related note, another proposal was to structure an SUS in such a way that creation of an action log can also be automated. Would anyone in the community like to help tackle these challenges?
Furthermore, the group discussed how usage scenarios could also be repurposed for testing responsiveness and general performance on older or low-end hardware. What are your ideas?
Did you know we are planning to have a measure-athon to measure the energy consumption of FOSS/KDE applications in early 2022? Which applications would you like to see Standard Usage Scenarios prepared for? Share your ideas with us here or submit your SUS at our GitLab repository!
Replicable reference system
Standard Usage Scenarios are run on a reference system, and an important issue raised was how to have a replicable reference system. There are several system layers to consider: Hardware (CPU, Disk, Ram, etc.); the OS and OS services such as the activity manager in Plasma as well as their configurations; and the tested application's configuration as well as any related applications (e.g., Akonadi for PIM applications). On the one hand, it is necessary to have a controlled environment in order to be able to interpret the results; on the other hand, it is desirable to have measurements which reflect actual, perhaps even noisy computing environments in order to understand the real-world energy consumption of software. In the end, whether to have controlled or natural computing environments will depend on what the data is to be used for, and this will differ for researchers, developers, eco-label auditors, companies, etc.
For instance, for developers interested in doing energy efficiency unit tests or obtaining the Blauer Engel seal, it will be necessary to have clearly-specified environments. So what are convenient and quick ways to bring the system to a known state? Three ideas were discussed: (i) taking snapshots of a filesystem (e.g., Snapper), (ii) clearing system cache, and (iii) replacing configuration files. If you have other ideas let us know about them here!
For real-world environments, one proposal, potentially controversial, is to log hardware performance and electrical power usage while people use their personal computers. Any volunteers?
Data output
When measuring energy consumption in a lab, in particular for Blauer Engel eco-certification, three types of data files may be generated: (i) a log file of actions (see above regarding automation of such log files), (ii) hardware performance data (CPU, RAM, etc.), and (iii) electrical power usage. Since in some cases the output data will need to be self-defined (e.g., timestamp loggers in xdotool scripts), it is necessary to consider what the data should look like before measuring in the lab.
Several issues were raised related to the data: Should data output be standardized to harmonize output across measurement campaigns, or is it enough simply to publish the data collected in an accessible format (for example, as .csv-files such as at the FEEP repository)? What timestamp exactness is needed (e.g., seconds vs. fractional seconds)? These questions remain open for now, but the group decided we need an overview of the FOSS-compatible devices/scripts (e.g., Achim Guldner's script for Gude Expert Power Control 1202 Series) and toolkits (e.g., statistical analysis tools such as OSCAR, R, Julia, Octave, PSPP, NumPy) used for measurements and analysis, which is currently in progress.
Power meter devices
The Blauer Engel criteria refer to two power meters (PM) in particular: Janitza UMG 604 and Gude Expert Power Control 1202. These devices are not cheap. In 2020, long-time KDE contributor Volker Krause hacked an 8 EUR power plug and wrote about it. As described in the blog post, these inexpensive power plugs are able to get a sampling rate up to 200ms. This led to the idea of using inexpensive power meters in low-budget home labs. For this purpose, it will eventually be useful to compare results from professional power meters with budget ones to see how they compare. Would anyone in the community be interested in doing this?
From Volker Krause's Blog Post: "Example measurement (200ms sample interval, no filter): mouse cursor being moved between samples 100 and 150, with a peak while hovering a task bar entry."
Another way to measure energy consumption came from KDE contributor David Hurka, who has a proposal for USB SPI power sensors to provide a higher resolution alternative to power meters measuring at the wall outlet.
Open issues for measurement labs
Several other issues were discussed related to energy consumption and lab measurements. Although final decisions were not made in every case, many of these open issues will be important to consider going forward.
- Idea of a marketing-friendly tool to draw attention to the energy consumption of software.
- For instance, an 'Eco' button for energy efficiency/carbon footprint meter (cf. travel carbon footprint in Itinerary app).
- Note that a similar functionality is already provided by PowerTop, though open issues remain (when is it safe to enable to avoid data loss, machine freezes?).
- When do the drawbacks outweigh the benefits of support for older hardware?
- Let's invite researchers working on this to discuss with the KDE Eco community.
- Future collaboration:
- Joint Sprint with related projects (e.g., SoftAWERE)? (Perhaps of interest to the reader: Max Schulze from SoftAWERE/SDIA presented at KDE Eco's monthly meetup in January 2022, which you can read the minutes from here.)
- Discuss above topics and more with other related communities (Bits & Bäume, Forum InformatikerInnen für Frieden und gesellschaftliche Verantwortung, etc.)?
Conclusion
The online Sprint was a wonderful opportunity to bring the community together and move the KDE Eco project forward, especially as we prepare for the community measurement lab that will be held at KDAB Berlin and the first of many measure-athons (planned for early 2022)! The success of such events depends first and foremost on the community, so allow us to send a heartfelt thank you to everyone who joined the conversation. We look forward to making the community grow in 2022! Moreover, we could not do what we do without the support of KDE e.V. as well as BMU/UBA, who financially support the BE4FOSS project (although only the publisher is responsible for the content of this publication).
Funding Notice
The BE4FOSS project was funded by the Federal Environment Agency and the Federal Ministry for the Environment, Nature Conservation, Nuclear Safety and Consumer Protection (BMUV1). The funds are made available by resolution of the German Bundestag.
The publisher is responsible for the content of this publication.
1 Official BMUV and UBA-Logos are sent only by request at: verbaendefoerderung@uba.de