Our team and mission

This position will be located at the European Astronaut Centre (EAC)

Within the XR-Lab, you will work on Virtual Reality applications for astronaut training and space exploration. The potential usage of XR (Virtual Reality, Augmented Reality or Mixed Reality) and associated haptic devices at EAC are suitable for EVA, robotics, LUNA analogue facility and onboard ISS purposes requiring incremental preparation, prototyping and tests of tools helping the crew and ground support personal to design, prepare or complement space training. XR at EAC is also used in support of mission design for various missions (Low Earth Orbit, EL3 on the Moon, Gateway around the Moon or Mars). When used in weightlessness conditions, XR hardware and software necessitate specific research, modification, tests or ground up design to ensure full realism, functionality, and robust tracking in Zero-G conditions.

Therefore, the XR-lab is looking for 3 candidates:

1. In the first project, you will be familiarized with XR-Lab’s project portfolio and will be asked to use their knowledge to improve existing 3D models and their design workflows, as well as to create new models and workflows as required.
2. The second project is software based and intends to improve various tools already developed at the XR-lab (JIVE, LUNA) to support and optimize their transition to MR.
3. The third project is more dedicated to AR, where computer vision techniques need to be further developed towards the implementation of a Medical Assistant for astronauts.

All 3 internships will be also focused on analyzing the current tools and XR-Lab on-going projects, complement them (Unreal Engine based for MR and Unity for AR) and create/modify relevant additional tools or models in close cooperation with the XR-Lab team.

For further information visit our web site: http://www.esa.int

Field(s) of activity for the internship

You can choose between the following topics:

1) Topic 1: Optimization of 3D-Modelling, Rendering and Design Workflows in XR Applications for Space Exploration

The XR-Lab at EAC engages in a variety of projects to promote the use of immersive technologies in space exploration, frequently partnering with international collaborators. These endeavours often require working with very highly detailed and industry grade 3D models.

This position will focus on refining and enhancing the models used in our applications, with a specific focus on improving modelling workflows and rendering techniques. This provides an opportunity for you to significantly contribute to various XR-Lab projects. Currently, the software has been delivered to NASA and is in use for Astronaut Training. ESA is using JIVE in 2023, combining JIVE and the classic Dynamic Skills Trainer (DST) at EAC. LUNA is a building under construction across EAC and has been virtualised in VR.

As an intern, your responsibilities will also encompass both the implementation and evaluation of rendering optimization strategies, as well as the exploration of innovative rendering technologies (DLSS, LODs, Nanite, etc.). These combined efforts aim to substantially enhance the performance and visual quality of our XR applications.

2) Topic 2: Support the transition to MR (Mixed Reality) for training, design and operational tools.

The second project intends to improve various software and assess the transition to Mixed Reality (MR). The Joint Investigation into VR for Education (JIVE) is a VR application that is currently partially testing MR. JIVE explores the best combination of tools to aid robotics instructors deliver astronaut lessons. It is a collaborative project between EAC’s XR Lab and NASA JSC.

.

Currently, the software has been delivered to NASA and is in use for Astronaut Training. ESA is using JIVE in 2023, combining JIVE and the classic Dynamic Skills Trainer (DST) at EAC. LUNA is a building under construction across EAC and has been virtualised in VR. However, MR has been explored to support the outfitting phase of the building.

This current position will therefore improve JIVE following training feedbacks of astronaut/trainers, check the current MR usage and as part of necessary improvement process, improve MR usage. For LUNA initial tests in MR have been successfully conducted and further implementations will be necessary such as hand free UI, menu optimisation and tracking solutions. Other projects in development at XR-Lab should be also supported when MR is considered.

The project aims at deploying all MR projects on Varjo XR-3 devices.

3) Topic 3: Computer vision expert in the support of the Medical Augmented Reality Assistant

Technological advancements in the areas of AR have created new possibilities for terrestrial learning and training, allowing information to be presented in different ways and releasing users from the need to see a display screen to retrieve information by allowing them to look directly at the object(s) of interest. As such these technologies provide an opportunity to develop ‘advanced’ procedures for the operation of medical devices. Such an approach has the potential to increase the quality of procedures, both in terms of the delivery of information and ease with which they can be followed by a crewmember with limited or no medical knowledge or experience.

To enable such solution, we propose to focus this position on computer vision supporting the development of MARA (Medical AR Assistant).

• Making use of computer vision to develop a software solution supporting the design, development and validation of a medical AR assistant.
• Prototypes of such software has begun at ESA XR-Lab, further developments are needed to be fully integrated in Hololens 2 (body parts recognition, pose acquisition for calibration, subject tracking and AR guidance).

Behavioural competencies

Result Orientation
Operational Efficiency
Fostering Cooperation
Relationship Management
Continuous Improvement
Forward Thinking

Education

You must have student status and be enrolled at university for the entire duration of the internship. You should preferably be in your final or second to last year of a university course at master’s level in a technical or scientific discipline.

Additional requirements

The working languages of the Agency are English and French. A good knowledge of one of these is required. Knowledge of another Member State language would be an asset.

Additional requirements:

1) Topic 1:

3D Modelling and Rendering Optimization:
• Core responsibility: Ensuring models are optimized for interactive real-time XR usage.
• Strong background in 3D-Modelling and Texturing
• Ability to accurately reconstruct 3D objects from technical drawings, CAD models, photos, and videos.
• Knowledge of rendering pipelines and systems
• Knowledge related to XR middleware such as Vulkan, OpenVR, and OpenXR are considered a plus.
• Knowledge and skills in VR/AR development (or better MR)
• Experience with Game-Engines (ideally Unreal/Unity) or other relevant tools.

2) Topic 2:

• Knowledge and background in skills relating to MR development
• Relevant programming capability in this area and with relevant tools (e.g. Unreal/Unity)
• Proficiency in the use of Microsoft Office
• Once the MR improvements are implement and MR has been implemented in JIVE or LUNA projects, the intern shall further interact with the rest of the XR team to best test the device with other MR projects.

3) Topic 3:

• Computer Vision software development
• Embedded systems design (UART and SPI interfaces)
• ARM-based software development
• Signal processing and Control Theory (Kalman filtering, digital filter design, sensor data fusion)
• In addition, knowledge related to XR middleware like Vulkan, OpenVR, and OpenXR are considered a plus.
• Prior development experience with Nvidia Jetson products is also desirable.