LONG READ
Artists and creatives not only touch, entertain, and amaze with their works but also explore and comment on the impact of scientific findings and new technologies on society and consider alternative developments. These examinations involve the negotiations on values and addressing incomplete knowledge. How do we want to live together? What decisions must be made despite only partially understanding their effects? How can we create spaces where discussions on values can occur early, allowing citizens to influence technological developments? The challenge lies in comprehending new methods and scientific developments and their effects, ideally enabling many to contribute to their form. It is not just about speculating about future technological applications; it is about determining who will be involved in developing these new methods and procedures. New scientific findings and their resulting technological developments can be assessed differently depending on their impact on an organization, society, or individual. What constitutes relevant, innovative developments in science and technology? Furthermore, what challenges and ethical questions do individual scientists and developers face? Whom will we empower to find answers to the questions they consider relevant? What are the social and ecological effects and ethical consequences? These central questions, posed by artworks, add value by promoting interest in and stimulating discussion of the social implications inherent in the development of new technologies and the STEM science behind it.
In this overview article, I provide examples of multi- and transdisciplinary transformative spaces and the requirements for creating them to address these issues. I give insight into artistic projects and the creative practices of the scientists and into the joint artist-scientist projects integrated into various working fields within an applied science research organization. These exemplary projects aim to foster innovative public engagement and scientific teaching and learning with new stakeholders, promote self-driven human resource development, and try out new ways to deal with central research and development questions and tools. They empowered both employees and partners to expand their areas of responsibility and take ownership of their actions. Artistic works can facilitate access to highly complex technology and science, build bridges for a broader audience, and facilitate exchanges among affected stakeholders. Artistic and creative practices are increasingly entering the realm of research and development (R&D) to tackle complex issues and address social, environmental, and economic challenges in new ways. The digital age has led to a trend towards the scientification of art. Researchers, developers, artists, and creators increasingly use similar tools, methods, and software. Scientists use creative or even artistic practices, but these are not necessarily considered in organizational self-conceptions or working procedures. Art already plays a role in exploring new technologies, applications, and possible societal challenges. Additionally, creative and artistic practices allow critical reflection, ignition of inner motivations, discovery of talent, and fostering of participation in collective action and a sense of belonging. Art can help employees and citizens take an interest in new developments, encourage individual responsibility, and strengthen their abilities to adapt. Through aestheticization and co-production, Sci-Art projects allow positive connections to STEM sciences, facilitating multidirectional dialogue, knowledge transfer, and communication.
Creating the Preconditions and Contexts for Temporary Transformative Spaces; Example Fraunhofer MEVIS
The Fraunhofer Institute for Digital Medicine MEVIS, comprising approximately 150 staff members, is dedicated to advancing digital medicine. Scientists at Fraunhofer MEVIS have addressed the growing complexity of healthcare by designing and developing software solutions that enhance data integration and interpretation. As the understanding of pathologies expands, new diagnostic and therapeutic tools have emerged. The resulting multidimensional and multidisciplinary processes for diagnosis, therapy decisions, and monitoring must be handled efficiently as part of a cost- driven healthcare system.
Medical science is systematized knowledge that sheds light on how conclusions are reached and identifies potential sources of error. However, it is crucial in medical science to determine whether this knowledge is reliable. The consistent management of sources of error in humans and technology (including AI) will be indispensable in allowing systems to increase safety, productivity, and quality in medicine. To achieve this, certain prerequisites must be in place, including a positive error culture, robust and inclusive governance structures in participatory working environments, multi-and transdisciplinary cooperation, and international networks.
MEVIS offers a work environment that values lateral leadership, shared responsibility, and participation. The aim is to cultivate high engagement and individual entrepreneurial behavior, fostering success within the organization. Researchers come from a wide range of STEM disciplines and have the freedom to work autonomously within a self-organized dynamic network. Project teams are formed to include members with diverse technological and clinical backgrounds, allowing them to leverage the collective expertise and experience of each individual. Non-hierarchical, dynamic collaboration promotes individual multidisciplinary training and cooperation, benefiting employees and the organization. Internal communication at MEVIS is governed by transparency to empower employees and harness their potential for growth. All team members are welcomed in any meeting that interests them, and all are granted access to all documents and business information. Restrictions apply only because of legal constraints or confidential agreements between partners and customers. Information sharing is expected and facilitated through exchange forums, such as wiki-based intranets. Additionally, self-motivated initiatives from staff members beyond their current assignments are encouraged.
These fundamental principles are reflected in participatory science-art projects realized at the Institute. These projects contextualize current research results and create science-informed artwork, emphasizing learning and science engagement. They also provide researchers with a means to address their work's broader impacts and social dimensions. Within these innovative projects, artists and scientists seamlessly transition between roles, serving as consultants, co-creators, research partners, and even STEAM (science, technology, engineering, arts, and mathematics) educators for students in schools.
Socially relevant and complex issues, such as accessible and affordable healthcare, environmental and climate protection, and lifelong learning, require reconsidering priorities in developing product-ready solutions and services. Success can no longer be solely defined in terms of profit and efficiency. The intricate crises at hand necessitate diversity, open dialogue, skill development opportunities, and community access to knowledge. How can new spaces for transformation that address these tasks into the structures, fields of action, and work processes of an organization be gradually integrated?
Integrated Artistic and Scientific Practices: Strategic Alignment for Meaningful Outcomes
The crucial aspect involves integrating Sci-Art projects with the objectives of the institute's overarching mission and the topics they tackle. This alignment ensures that the projects are directly anchored to the staff's expertise and activities in a participatory manner while also integrating new stakeholders. In doing so, joint artists’ and scientists’ undertakings can generate meaningful outcomes. The institute's team of scientists and creative partners collaboratively works to broaden the collective realm of solutions through mutual learning and creation. This collaborative effort extends beyond the organization and fosters integration within a specific sector. These principles have the potential to be tailored and applied across various industries and subject domains.
In the following diagram, the x-axis represents the degree of alignment with the principles, methods, procedures, and tools used to achieve the main missions of an organization. The y-axis indicates the artist's or creator's involvement with the organization’s specific objectives. As an illustration, apart from to a company's main mission, attention could be directed towards human resources, diversity, or sustainability aims and missions. Fraunhofer MEVIS has focused on 1) science PR, experiential, aestheticized science, 2) participatory science engagement, co-production, and knowledge transfer (summarized as strategic R&D engagement), and 3) the R&D questions and developments driven by the institute's primary mission. At Fraunhofer MEVIS, R&D engagement and cooperation with artists and filmmakers are considered unique opportunities. It lets us reach new and diverse audiences, convey significant advancements, facilitate emotional and positive encounters with science, create contemporary science engagement formats, and also allows our staff to explore interests beyond their daily work.
In the beginning, Fraunhofer MEVIS lacked a structured program for Sci-Art projects within the research institution. Instead, discussions were held with scientists who were either enthusiastic about pushing the boundaries of their research tools or were creative drivers of public engagement. The residency program, STEAM Imaging, was created to align with scientists' expertise and interests. It facilitated the development of Sci-Art projects, established research collaborations with the artists, and made a collaborative learning room that deviated from typical procedures and workflow. Artistic practices possess the capacity to unveil innate talent and interest. Getting creators on board, fostering multidirectional models of knowledge transfer within an organization, and developing joint projects with additional stakeholders can refine and enhance a company's mission. This approach allows to perceive staff and partners as active citizens, while also recognizing an institution or company as a powerful force for addressing socially relevant topics.
Addressing Research & Developments: Bringing Relevant Topics to Unusual Forums, Making them Accessible to a Broader Public
Main Mission Integration: Medium
R&D Engagement Integration: Medium
Science PR Integration: Medium to High
Realized Formats: Interactive exhibits, immersive audio-visual 2D, stereoscopic 3D, and hemispheric installations
Specific Aims: Addressing actual R&D technologies, procedures and topics in digital medicine. Reaching non-tech-affine audiences, allowing them to build emotional positive relations to STEM & tech and topics around severe diseases. Approaching digital medicine through artistic lenses.
Projects falling under this category leverage and/or grant access to outcomes and technologies that align with the research objectives of the institute's primary mission. Based on real medical images and data, scientists used 'MeVisLab' (a software platform they usually use to develop assistant tools for clinicians) to create immersive audio-visual 2D, stereoscopic 3D short films and large-scale hemispheric installations. Adopting an artistic perspective when examining medical data enabled them to bring abstract concepts and severe health topics into a tangible realm to the public and new places. Additionally, aesthetic elements and artistic approaches can influence researchers' reflections on work topics.
Artistic perspectives and innovative research approaches necessitate the ability to literally play with technologies, while leveraging expertise and domain knowledge in an inclusive and collaborative environment. The roles of the leader, creative driver, giver, and learner are fluid. Simultaneously, it remains essential to be mindful of the context for which the material is produced and to investigate the occasion, intended users, or audience. See case studies 1 and 2.
Approaching R&D Topics Through Artistic Lenses and Practices: A Culture and Mindset of Collective Sense Making
Main Mission Integration: Low to Medium
R&D Engagement Integration: Medium to High
Science PR Integration: Medium to High
Realized Formats: Creators Residencies; outcomes include: audio-visual installations, AR sculpture, sound device development, sound installations, live performances
Specific Aims: Creating engaging artworks. Approaching digital medicine with artistic practices, engaging the next generation in R&D topics and tools,
discussing actual challenges in digital medicine, and
experiencing STEM and tech with awe and fun.
The goals of the creators’ residencies range from engaging staff in arts-based training to
sustained dialogue with the broader public to develop and co-create collaborative projects with scientists, the next generation, and the community to establish new multi-stakeholder R&D spheres. Artists can also drive innovation in scientific engagement. Artists entering a collaborative realm within the research department are subject to specific requirements and prerequisite knowledge. In all cases, it is essential to refrain from instrumentalizing artists. To ensure that the artists' strategies flourish within these collaborations, they do not have to adapt their thinking and processes to those of technology, business, and communication paradigms.
The international creator-in-residency program 'STEAM Imaging' at Fraunhofer MEVIS, which runs for the fifth time in 2023, was designed to fuel contextualization, self-efficacy, and diversity in STEM and digital medicine with a strong focus on science-inspired or science-backed artwork development and collaboration with future generations. Autonomous and cross-generational learning will gain significance. STEAM Imaging is a residency program that unites applied research, academia, and schools as the stakeholders of shaping the future of medicine.
New contexts for learning and deep interdependent thinking are needed to discover meaningful answers in healthcare and scientific education in the future. In the landscape of digital medicine, there is an increasing array of intricate challenges. Like all real-world problems, including climate change, zoonotic diseases, and social inequality, solving them requires the integration of many disciplines. Despite concerted efforts to mend divisions between disciplines, they remain problematic. This sense of division is evident in educational institutions, professions, business models, funding structures, and the industrial sector. However, these divisions do not help solve complex issues sustainably, and some questions remain. How much depth do we have to sacrifice in favor of interdisciplinary breadth? How deep of an understanding of basic concepts in physics, biology, mathematics, or computer science is needed to apply them in a connected, sensible way?
STEAM Imaging allows artists to exchange information intensively with scientists to critically examine current methods, developments, and research results in their work to create their own artwork. A pivotal element of the program involves shared interactions with school students. The art-and-research alliance enables prospective university students to approach scientific and technical topics from new and unexpected perspectives. It encourages them to engage with science, technology, and art in a self-determined and creative way. At the same time, non-tech-savvy audiences is addressed through the artworks exhibited.
Varied perspectives play a critical role in application-oriented development within R&D institutions. Multidisciplinary and transdisciplinary collaborations between artists and scientists create spaces for dialogue, which can lead to new ways of thinking about a subject. Tailored creator residencies can (within a guided process) enhance the potential of companies or organizations to adapt to social, environmental, and economic challenges. By engaging with communities and assuming an influential role in society, they can accelerate and enhance companies' outcomes, encompassing strategic engagement, communication, R&D, services, and products. The goal is to contextualize a company's mission into a broader ecosystem of stakeholders that build emotionally positive and critical relationships with new technologies, developments, and products and services. The case study 'Whose Scalpel,’ arising from the artist-in-residence program 'STEAM Imaging I,’ exemplifies how questions relevant to society and profoundly intricate, abstract, and serious subjects can be made more accessible to a broader audience. See case study 3.
STEM+ and Extracurricular Places of Learning: Integrating Artistic Approaches in Scientific Teaching and Learning with STEAM Workshops and Courses
Main Mission Integration: Low
R&D Engagement Integration: Medium to High
Science PR Integration: Medium
Realized Formats: STEAM workshops & courses, STEAM evenings with school student's family & friends, web-based learning tools, teaching material, incorporating real work tools, procedures, and real medical data.
Specific Aims: Engaging with R&D topics and tools, discussing current challenges in digital medicine. Experiential STEM+. Approaching digital medicine with artistic practices.
An integral part of the STEAM Imaging residency program is the participatory STEAM courses and workshops held jointly with artists, students, and scientists. Artistic practices and thinking fuel the understanding of complex connections from multiple perspectives and act as translators between different stakeholders and communities of knowledge.
Moving successfully between various disciplines is a crucial competency for future education and innovation. STEAM courses can enhance multidisciplinary talent, inspire students to explore innovative educational models, and foster self- driven engagement with art, science, and technology, leading to the acquisition of multidisciplinary skills. This has the potential to broaden students’ creative practices and expressions, offering valuable advantages in the future job market. At the core are participatory inquiries, which include both fundamental knowledge and free creative exploration, to help students understand a subject thoroughly. Beyond students, this concept can be customized to involve the internal workforce, supply chain partners, and even consumers aligned with their respective sectors, technologies, processes, and tools. The idea behind such a joint project-oriented creation and learning is not to impart a canon of knowledge. It serves to explore an issue as an example of learning how to autonomously develop innovative solutions, navigate new technologies, and facilitate critical discussions on these subjects. Interacting playfully with core concepts, technologies, and tools (which are crucial for a specific product or service) allows companies and participants new avenues for viewing and dealing with real-world questions and challenges. In STEAM courses, participants explore topics, theories, real-life methods, and practices. These courses are versatile, interdisciplinary, and cross-generational platforms designed to tackle the implications of emerging technologies in their early stages. Through the joint development of courses by staff and creatives, the roles of teacher and learner become fluid. STEAM workshops and courses provide diverse role models and can assist school students in establishing valuable connections between education and career opportunities, ones they might not have contemplated before. Leveraging new technologies and tools can be a true eye-opener for many participants.
Upcoming generations will engage with tools and professions that might currently be beyond our awareness. Therefore, it is crucial to engage schools, teachers, and communities in collaboration and channels involving companies working alongside students. This approach allowed us to explore issues, raise questions, and discover answers that transcended disciplinary boundaries. The basic comprehension of new technologies should be a part of everyone’s education. Owing to the speed of development, this can no longer be achieved solely by adults. Cross-generational dialogue is required, particularly in education. How can communication and knowledge flow bidirectionally? STEAM courses, tools, and creator residencies involving the younger generation hold promise as potential solutions for addressing these challenges. See case study 4.
Contributing to Research and Developments: Artistic Practice Focusing on Programming Interfaces
Main Mission Integration: High
R&D Engagement Integration: High
Science PR Integration: High
Aspired Format: Tool for intuitive MRI sequence development
Specific aims: Make expert tools accessible to a diverse range of users
Currently, MEVIS scientists are seeking funding to integrate artistic practices within a research project focusing on programming interfaces to develop magnetic resonance imaging (MRI) sequences broadening diverse access to expert tools and knowledge. In the future, the intuitive and user-friendly interface should also enable research clinicians and medical staff to explore new imaging ideas quickly and efficiently. Furthermore, this realm of expertise will be accessible to a broader array of professions and usage contexts, including art and artwork dvelopment. Clinical staff, teachers, students, and creatives will have the opportunity to explore sequence development ideas using a basic version of the expert tools. This shift enables a more open, creative approach to MR sequence development, opens up broad access and knowledge transfer to society, and, ideally, pushes the limits of what can be achieved in medical imaging, enhancing options for patient examinations. See case study 5.
Supporting Creators in Scientifically Inspired Artwork Development
Main Mission Integration: None
R&D Engagement Integration: Low
Science PR Integration: Medium to High
Realized Formats: Audio-Immersive and VR Experiences
Specific Aims: Raising awareness about possibilities in digital medicine. Looking at digital medicine from an artistic perspective. Bringing engaging artworks to new public places, reaching broader audiences.
In addition to their artistically oriented works, MEVIS scientists also provide support or consultation to national and international artists and engage in collaborative projects. R&D partnerships for artwork development are an excellent way to collaborate with artists. The scientists provide access to expert tools and knowledge and, for instance, process and prepare data; mutual knowledge exchange allows the creation of bridges between the sectors of artwork creation and research and development. The resulting exhibitions and, for example, accompanying educative events can serve aspects of the mission of an organization. An illustration of this is the collaboration between Fraunhofer MEVIS and the London-based artist collective Marshmallow Laser Feast, which yielded the artworks 'The Tides Within Us' and ‘EVOLVER.’ The autonomy of the artists, their expertises and approaches which deviate from processes in R&D and business, was crucial; it was important that they took their perspectives onto the methods and technologies in digital medicine. At the same time, it was essential that they were attentive to the scientific background without diluting the precise and comprehensive knowledge possessed by the scientists.
In the field of medical technology, researchers are accustomed to operating in inter-, multi-, or transdisciplinary manners. Engaging in collaboration with artists opens up diverse avenues for exploring and presenting scientific data and topics. The artists expand the pathways through which scientists, and ultimately the broader public, can perceive and address phenomena concerning the human body, along with the questions and challenges posed. See case study 6.
SELECTED SCI-ART CASE STUDIES AT FRAUNHOFER MEVIS
1 'The Beauty of Blood Flow Analysis', an immersive audio-visual 2D, stereoscopic 3D, and hemispheric installation, conveys emerging possibilities for gaining knowledge and making predictions about the human body
‘Great visualisation, sophisticated, cutting edge technologies, and fore and foremost clinical valuable information.’ Dirk Simon, Siemens Digital Factory
Artistic, spacial, immersive experiences enable emotional, positive engagement with severe health topics and new developments in digital medicine. Fraunhofer MEVIS scientists, somewhat like artists themselves, creatively applied diagnostic technology to create 'The Beauty of Blood Flow Analysis’. The short piece is based on new research and technologies in digital medicine. It conveys emerging avenues for acquiring knowledge and making predictions regarding the human body. They strictly and deliberately confined themselves to real medical data used for diagnostics, treatment planning, and monitoring. The project's aim was to reach a broader, not necessarily tech-affine, audience on an emotional level. In developing software assistant tools for clinical routines and producing exhibits and moving images for science engagement, the MEVIS team witnessed the enormous beauty of blood flow. They naturally wanted to stimulate discussion around treating severe diseases, for which they are developing new procedures and technologies, thus introducing them to a societal, cultural world. The project allows the audience to witness the beauty of the swirling flow of blood in the heart and gives an idea of the awe-inspiring steadiness of the heartbeat. By observing flowing blood, various diseases can be detected. As an illustration, turbulent vortices indicate a malfunction of the heart valve. Software assistant tools help clinicians navigate and integrate this complex information. Medical data is a sensitive topic; and finding a line in the technical production that still provides enough anchors for discussing real-world clinical problems, as well as providing enough space for awe-inspiring artistic visuals, was a great challenge for the scientists. While retaining this authenticity through Sci-Art, the distance between the viewer and the scientific medical work is bridged. The project ‘Beauty of Blood Flow Analysis' has been the point of departure for further explorations at the institute together with artists. The goal is to add educational programmes to scientifically informed art installations and engage diverse audiences in the wonders of the human body and the constructive possibilities of new technologies. The prize-winning work, 'The Beauty of Blood Flow Analysis’, is shown in international venues in broad contexts, such as conferences on healthcare technology, in science centres, and at science, tech, and art festivals. (See addendum 1.1).
2 'Digital Medicine, Arts, and STEAM: Before Us Lies ETERNERDY’, an immersive audio-visual 2D and stereoscopic 3D installation, on a technical, scientific, and mathematical understanding of the human body
‘The ETERNERDY, coined from the idea of “The Eternal Nerd,” is far more than a mere visual spectacle. It is a symbolic testament to the limitless bounds of human curiosity. Not simply a study of art or science, ETERNERDY delves into the essence of human creativity and the quest for knowledge. It unravels the complexities of digital medical data, and brings intricate discussions on severe diseases and new technological possibilities to diverse audiences. By leveraging art as a medium, this film demystifies severe health topics and encourages public engagement.’
Ina Conradi, Artist, Associate Professor, The Nanyang Technological University
'Digital Medicine, Arts, and STEAM: Before Us Lies ETERNERDY’ by Fraunhofer MEVIS scientists, gives insights into several ways to interpret the human body with different kinds of data acquisition and processing. Understanding and negotiating complex issues in digital medicine requires expertise, time, and dedication. But who is developing new technological possibilities, telling their stories, and shaping the medical future? The immersive installation is based on reflections about researchers and their curiosity and creativity in dealing with issues that can only be penetrated through technical, scientific, and mathematical understanding. The installation shows a floating human body in a space-like environment. Accompanied by spheric vocals, it looks into the body and an organ down to a layer of tissue that, when magnified, reveals itself as the space environment with star-like lights in which the body floats. The project shows different scales of the human body: from digitised microscopic lymphoma tissue examined with the molecular cytogenetic technique fluorescent in situ hybridisation (that detects abnormal changes in DNA) to 3D reconstructions of two vessel systems of a liver (for patient-individual surgery planning) to a whole-body MRI. Fraunhofer MEVIS researches and develops MR sequences for clinical practice, software for automatic and precise analysis of tissue sections for digital pathology, and is a pioneer in combining and analysing medical data for better information integration and decision support for medical professionals. The internationally awarded work premiered simultaneously in 2D and stereoscopic 3D, bridging audiences in Austria and Singapore on-site with a live Q&A session with the scientists and artists involved. It was exhibited at various venues and festivals at the intersection of science, art, and technology. (See addendum 2.1).
3 'Whose Scalpel', a live performance integrating developed sound devices, addresses questions around decision support for medical professionals: what do humans lose when the computer takes charge in the operating room?
‘Linking science, technology, and art holds great potential for addressing our social, environmental, and economic challenges in Europe. 'STEAM Imaging' is a programme that has been unleashing this power for years, creating an international lighthouse project focused on knowledge transfer and learning through art. We need more programmes like this.'
Veronika Liebl, Managing Director Festival Prix Exhibitions, Ars Electronica
Do new developments such as artificial intelligence present opportunities and inspiration, or will humans left behind as casualties of progress? 'Whose Scalpel’ by artist Yen Tzu Chang addresses the complexity of art and medicine and possible consequences for our society. It asks what human beings lose when the computer takes over. The accompanying installation is based on a supersized 3D-printed model of the artist's heart, equipped with sound generators. During the performance, Yen Tzu Chang acted as a 'surgeon', placing a bypass with the help of cables – presumably guided by artificial intelligence. Through the almost complete darkness, an intense, unsettling backdrop is created, which only conciliates at the end of the performance.'Whose Scalpel' was created as part of 'STEAM Imaging I,' a residency programme to link science, technology, and mathematics with the world of art. The project combined computational medicine with STEM issues, sound art, and ethical discussions. During her stay at the institute in Bremen, Yen Tzu Chang learned how to use MeVisLab, a software platform for processing medical image data. MR imaging and MeVisLab were also central topics of the STEAM school student workshops, which are an integral part of the residency programme. The aim was to break down barriers between the disciplines and to explore flexible forms of learning and collaboration. 'Whose Scalpel' was exhibited at various international venues and referred to in expert publications. In October 2023, 'Whose Scalpel 2.0' will premiere, enhancing the installation and integrating a dancer into the performance. (See addendum 3.1)
4 ‘Inside Insight’, a web-based interactive STEAM application to create an understanding of the value of physics, mathematics, and computer science for medical imaging
‘I like to use Inside Insight in computer science classes as a treat at the end of the semester. Our learners are eager for real-world topics. They are fascinated by the medical images with which they can work interactively, guided by experts. It is crucial to convey how important it can be for all of us to use algorithms intelligently and that their development is based on interdisciplinary expert knowledge.’
Susanne Pedersen, teacher for computer science and mathematics, Gymnasium Ulricianum Aurich, Germany
The tool serves as a foundation for grasping the basics of medical imaging and an idea of the capabilities of 'MeVisLab,' a rapid-prototyping platform that allows medical data processing. Typically, 'MeVisLab' is used by researchers and developers to build assistant tools for clinicians. The web-based tool 'Inside Insight' is created from materials and prepared use cases scientists have developed over the years to showcase possibilities of applications in the medical routine. MEVIS scientists aimed to make real-world tools (developed with 'MeVisLab') and explanatory materials easily accessible in an explorative and creative way. Users simply log in via a web browser. The tool encourages users to explore medical images creatively, thereby playing with STEM knowledge. It is completed with explanatory material, enhanced by an artist, and audio files. 'Inside Insight' helps users understand that questions about the body can be answered by mathematics, physics, and computer science. The tool was nominated as a finalist in the category Best Educational Media at an international science film festival and is in regular use in numerous workshops and courses in the field of STEM teaching and medicine and art and technology alike. (See addendum 4.1).
5 'gammaSTAR' and 'gammaSTAR element’
Magnetic resonance imaging is an essential medical diagnostic tool. The image acquisition process, unlike with X-ray, is complex and requires special software to control the MRI sequences. Traditionally, developing such sequences has been the domain of a small group of highly specialized software engineers. Through gammaSTAR, MEVIS scientists will empower a wider array of stockholders to develop MR sequences – vendor independent - and suitable fror multiple scanners from different manufacturers; gammaSTAR will not only expand the possibilities for exploration and experimentation in developing MR sequences, but also complement established procedures and protocols.
6 'The Tides within Us', an immersive interactive audio-visual installation, and 'EVOLVER', a VR experience on where the human body begins and where it ends.
'A mesmerising representation of the human body… It's a stunning work of exquisite educational power.’ Cool Hunting
Stunning interactive screens allow the audience to explore the human ecosystem. ‘The world flows into you, and you flow into the world’, describes artistic director Barnaby Steel, founder of artists collective Marshmallow Laser Feast. For the project, STEM experts had to find ways to give the artists medical anatomical and flow data in a technically consistent format from their perspective, which diverged from the manner in which these data are collected and processed in medical routines. The scientists had to create an understanding of what it represents and where the limits are, for instance, in accuracy and resolution. This collaboration has opened new ways of seeing and experiencing the human body. Together, the partnership aims to offer new platforms for experiential and embodied learning. The ultimate objective of the project is to reshape how people learn and think about themselves in relation to the environment. Where does their human body begin, and where does it end? 'Many different dynamic and complex processes occur within the human body every second. Gaining a deeper understanding of the details and variations of those processes helps us to understand diseases and optimize therapies. Measurement and imaging methods are utilised and developed every day. Even if those sensors are only capable of capturing a fraction of these processes, they generate a lot of complex data. To gain knowledge from this data, it needs to be understood and analyzed by experts. Visualization of this medical data is an important step in understanding. 'The Tides Within Us' is also a form of medical visualization, but with different goals and a much broader audience, who might not be medical experts’, explains Matthias Günther, Deputy Institute Director, Head of MR Physics, Fraunhofer MEVIS. The experts were challenged to reconsider the constraints of MR acquisition, medical data preparation, accessibility and provision of meta-information for non-MR scientists. Additionally, software architects gained a further understanding of tools used within the artistic community, such as cross-platform game engines and 3D animation software, and created bridges and workflows between these tools. The art also increase public and science engagement and appreciation of science and research. The work was presented internationally at art galleries and scientific conferences and won an industry award for best new media format and several honorary mentions as a unique and pioneering artwork. (See addendum 6.1).
ADDENDA: SELECTED HIGHLIGHTS OF EXHIBITIONS, SCREENINGS, PRESENTATIONS, AND PERFORMANCES
1.1. 'The Beauty of Blood Flow Analysis'
Screened at: Sparks! at CERN Future Technology for Health, 2022 (CH); Bright Festival Connect, 2020, Leipzig (DE); 9th International Festival of Science Visualization IFSV Dome Fest in Tokyo, 2019 (JP); STEAM art, Global LEAD Event World Tour New York City, Stanford LEAD Program, 2019 (US); 12th Fulldome Festival, 2018, Jena (DE); among others. Selected Highlights: 'Beauty of Blood Flow' was nominated as a Finalist at Falling Walls Science Breakthroughs of the Year 2022 in Art and Science, Berlin (DE). The Sci-Art work was recognized by the scientific documentary industry and won the 2018 Industry Award for Best Immersive Media at the Raw Science Film Festival in California (US). The work was nominated at the 12th Fulldome Festival, 2018, Jena (DE).
Credits: Directors and Science & Art Producer: Bianka Hofmann, Alexander Köhn, Mathias Neugebauer; Scientific Advisor: Anja Hennemuth; Sound: David Black; Fraunhofer MEVIS (DE)
2.1. 'Digital Medicine, Arts, and STEAM: Before Us Lies ETERNERDY'
Screened at Bright Festival Connect 2020, Leipzig (DE); CITY OF QUANTIFIED VISIONS, 2019 (SG), Raw Science Film Festival, 2019 (US), Ars Electronica Festival, 2018 Linz (A), Media Art Nexus NTU, Singapore, 2018 (SG); among others. Selected Highlights: The work was recognized by the scientific documentary industry and won the Industry Award for Best Infographic at the Raw Science Film Festival, 2019, California (US). The work premiered simultaneously at the Deep Space 8K at the Ars Electronica Center & Festival 2018 (AT) and at the Media Art Nexus (MAN) at Nanyang Technological University Singapore (SG) with a life connection and Q&A with audiences.
Credits: Directors and Science & Art Producer: Bianka Hofmann, Alexander Köhn, Mathias Neugebauer; Scientific Advice: Henning Höfener, Andre Homeyer; Sound: David Black; Fraunhofer MEVIS (DE). In cooperation with Ina Conradi and Mark Chavez, Media Art Nexus, Nanyang Technological University Singapore (SG); Fluorescent in situ hybridization (FISH) data kindly provided by ZytoVision; special thanks to Volker Diehl, Jochen Hirsch, Julian Haase, and Dagmar Weiß
3.1. 'Whose Scalpel'
Exhibited & performed at Science and Art in Dialog, 2018, Berlin (DE); FRAMELESS16, MUG at the Einstein, 2018, München (DE); Music Hackspace, Somerset House Studios, 2018, London (UK); Ars Electronica Festival, 2017, Linz (AT); among others. Selected Highlights: Hofmann, Bianka. (2019). Linking Science and Technology with Arts and the Next Generation—The Experimental Artist Residency "STEAM Imaging." Leonardo. 54. 1-10. 10.1162/ leon_a_01792. Schnugg, Claudia. (2019). Creating ArtScience Collaboration: Bringing Value to Organizations. 10.1007/978-3-030- 04549-4. Ed. Ars Electronica Center Linz, The Practice of Art & Science, The European Digital Art and Science Network (Hatje Cantz, 2017), pp. 50–55. 'STEAM Imaging I' was hosted by Fraunhofer MEVIS (DE) and Ars Electronica (AT) in collaboration with the International Fraunhofer Talent School Bremen; realized within the European Digital Art & Science Network
Credits: Artist & Sound & Performance: Yen Tzu Chang (TW), Fraunhofer MEVIS team: Bianka Hofmann, Sabrina Tölken (née Haase), Alexander Köhn, David Black. Ars Electronica team: Veronika Liebl, Jessica Galirow, Maria Pfeifer, Peter Freudling, Erwin Reitboeck. Technical support of medical images: Alexander Köhn. Technical support for 3D models and printing: Peter Freudling, Erwin Reitboeck, Benjamin Krux. Special thanks: Interface culture, Fabricio Lamoncha Martinez, Jie Ting Jiang, Yin-Wen Lin
4.1. 'Inside Insight'
The STEAM application is in regular use in numerous workshops and courses; it was used in the course 'Art and Medicine' at Paris Lodron Universität Salzburg PLUS & Mozarteum 2021 (AT), at Sci-Art NanoLab Summer School at UCLA in 2018 (US), among many other occasions. Selected Highlights: Schnugg, Claudia. (2020). Evaluation STEAM Imaging III: Art Meets Medical Research Online STEAM course 'Inside Out:10 STEAM Evenings' 10.13140/ RG.2.2.27760.38409. The STEAM application 'Inside Insight' was nominated as Finalist Best Educational Media 2018 at the Raw Science Film Festival, California (US) and in use within the graduate course by Prof. Roger Malina at UT Dallas, 2018, Special Topics in Arts Technology and Emerging Communication – Art, Health, and Medicine (US). Schnugg, Claudia. (2017). STEAM Imaging: Art Meets Medical Research: Evaluation Summary. 10.13140/ RG.2.2.11923.58403.
Credits: Fraunhofer MEVIS, with special thanks to the Young Talent Team. Realization: Bianka Hofmann, Alexander Köhn, Sabrina Tölken; in cooperation with artist Hannah Klatt
6.1. 'The Tides within Us' and 'EVOLVER'
'EVOLVER' was presented with Fraunhofer MEVIS as research partner at MUSEUM WAVE in 2023 in Seoul (KR), where Squid Game's Lee Jung-Jae joins Cate Blanchett in new voiceover for 'EVOLVER: A Virtual Reality Journey Of Life and Breath;' at the 2022 Geneva International Film Festival (CH), with the European Premiere of 'EVOLVER;' at the 2022 Tribeca Festival (US) 'EVOLVER – A Virtual Reality Journey of Life and Breath.' Presented at the launch program of The Reel Store, UK's first permanent immersive digital art gallery, opened in Coventry in 2022 (UK); SIGGRAPH Asia 2021 Art Galleries, 2021 (JP); Coventry City of Culture (co-commissioned and co-produced by Coventry City of Culture Trust and York Mediale, 2021); the Human Nature Exhibition at York Art Gallery, 2020/21; at Ars Electronica Festival 2020: A behind-the-scenes look and Guided Online Tour to the MR Lab by artist and scientists involved; an introduction to the artwork was given at First Star Scholars UK, University of Winchester 2021 by artist Barnaby Steel, followed by a hands-on online workshop from scientists on medical imaging based on the tool 'Inside Insight.' Selected Highlights: 'EVOLVER' received the Jury Mention for Storyscapes Award at Tribeca Festival 2022. 'The Tides Within Us' won the Industry Award for Best New Media Format at the Raw Science Film Festival, 2022 (US). The work found its way into the compelling book 'Imagining Imaging' by radiologist Michael R. Jackson. 'The Tides Within Us' received an Honorary Mention at the STARTS Prize 2021 as a unique and pioneering artwork. Credits: Concept: Marshmallow Laser Feast. Directed by: Barnaby Steel, Ersin Han Ersin, Robin McNicholas. In collaboration with Natan Sinigaglia. Scientific Partner: Fraunhofer Institute for Digital Medicine MEVIS. The Tides Within Us was co-commissioned by York Museums Trust, Mediale and Coventry City of Culture. Funded by Arts Council England.
Keywords: artiscience, artist’s residence; artistic practice; co-creation, computer graphics, creator’s residence; digital transformation, diversity; higher education; human resource development; innovation; interaction, knowledge transfer to the community; learning tool; media art; multidisciplinary, production; science communication, self-efficacy; skill development; sound art; participatory, Sci-Art; STEAM; experiential STEM; STEM+; strategic engagement; technology contextualization; transdisciplinary, visualization
I thank the curious and courageous scientists from Fraunhofer MEVIS’ R&D Engagement – doers, explorers, contributors – and the cooperating artists who made these explorations and projects possible. Many thanks to David Black for editing this English blog post.