Multiple PhD and postdoc positions on evaluation metrics for multimodal health data at the IT University of Copenhagen

The PURRlab (Pattern Recognition Revisited lab) at the IT University of Copenhagen invites highly motivated individuals to apply for a PhD or a 2 year Postdoc positions starting late 2025 or in 2026. The earliest possible start date is 1 October 2025.

The project is funded by the Novo Nordisk Foundation Data Science Ascending Investigator grant titled "CHEETAH: CHallenges of Evaluating Teams and Algorithms" and is led by Associate Professor Veronika Cheplygina.

Project description
Machine learning (ML) competitions are often touted as drivers of algorithm development in healthcare but face limitations in real-world applications. An example competition is detecting lung cancer in chest images, where the team correctly identifying the most images with cancer wins the competition. Such competitions attract many international teams with monetary or prestigious incentives. While competitions are said to spur innovation, they often result in too similar algorithms that only excel on a specific accuracy metric, but are not robust and fail to generalize to diverse, real-world data.

I posit that a single performance metric such as accuracy is insufficient to capture algorithm robustness, for example how the algorithm performs on rare patient cases. Having a single performance metric also leads to too similar algorithms which do not bring added value despite their high training costs and carbon footprint. Furthermore, as research on women and other underrepresented groups in computer science shows, competition may deter them from entering or staying in the field.

I therefore propose to design competitions with multiple metrics, both in what the metric measures (e.g. accuracy or sensitivity) and which subgroups of patients this is measured on. My team will focus on two multimodal disease (risk) prediction data: chest x-rays with radiology reports and retinal images with tabular clinical measurements. Inspired by techniques like generative models, transfer learning and data distillation we will first develop novel methods to evaluate and increase the diversity of the evaluation data. We will then design how to evaluate similarity of algorithms, and develop methods to combine and reuse (parts of) them, such that robustness can be increased without the disproportionate carbon footprint. Finally, we will organize competitions in education and at conferences, where we will study how the novel design affects underrepresented groups in data science.

The candidates
The project aims to hire 2 PhD students and 2 postdocs. PhD1 will work on chest x-ray imaging and NLP data, PhD2 will work on cardiovascular imaging and tabular data. The postdocs will both work across the applications, respectively on data diversity / algorithm diversity. All team members will be involved in the studies on competitions in research and education.

Ideal candidates for the positions will have:

- A finalized or close to completion MSc (for PhDs) or PhD (for postdocs) degree, in computer science, mathematics, cognitive science, human-computer interaction, mathematics education or another discipline with a strong computational component.
- Experience with data management and reproducibility, programming in Python or similar
- Experience with machine learning for imaging, NLP (PhD1), tabular (PhD2) or other types of multimodal data
- Experience with research related to education and/or involving human participants is advantageous
- For postdocs: experience with publishing in recognized journals, conferences or workshops
- Experience with co-supervising MSc/PhD students is advantageous
- Enthusiasm about interdisciplinary research in a diverse team

It is understood that even highly qualified applicants will satisfy these requirements to varying degrees, for example because of differences in countries and research fields. If you are interested in the project but are in doubt about checking these boxes, please contact Veronika Cheplygina before applying.

Research environment
The successful applicants will join the PURRlab team consisting of Associate Professor Veronika Cheplygina two postdocs and a PhD student working related projects within medical imaging and meta-science.

PhD1 (chest X-ray + NLP data) will be co-supervised by Rob van der Goot (ITU) and PhD2(cardiovascular + tabular data) will be supervised by Adam Hulman (Steno Diabetes Center Aarhus). Louise Meier Carlsen (ITU) will be involved as co-supervisor on the education part of the research.

The project also includes collaborations with Emory University School of Medicine (US), University of Bern (CH), Eindhoven University of Technology (NL) and the open-source platform OpenML.org.

There are multiple other opportunities for collaboration with other groups at ITU and a network of collaborations within and outside Denmark.

Here you can find some reflections by foreign PhD students from other research groups at ITU.

Data, Systems, and Robotics section
We are all part of the DSAR (Data, Systems, and Robotics) section at ITU. The DSAR Section at the IT University of Copenhagen conducts research in various aspects of infrastructure of data science, scalable and efficient systems supporting the data lifecycle, meta-science, artificial intelligence, robotics, and evolutionary computation, as well as a range of applications. Its faculty is responsible for many courses in the ITU BSc and MSc programs for Computer Science, Data Science and Software Design. The section currently has 12 faculty and 15 PhDs and postdocs.

General information
The IT University of Copenhagen (ITU) is a teaching and research-based tertiary institution concerned with information technology (IT) and the opportunities it offers. The IT University has more than 160 full-time Faculty members. Research and teaching in information technology span all academic activities which involve computers including computer science, information and media sciences, humanities and social sciences, business impact and the commercialization of IT.

Working at the IT University
Located in modern architecture, a 6-minute metro ride from the center of Copenhagen, the IT University offers an informal working environment, a varied everyday life, and a highly motivated, ambitious, competent, and diverse international faculty, with PhD degrees from 17 different countries. The IT University is an equal opportunity employer and prides itself in actively and successfully pursuing an improved gender balance among students and among faculty.

Working in Copenhagen
Copenhagen has a solid educational system, a rich cultural life, universal healthcare, good childcare, and well-functioning infrastructure. Copenhagen is among the world’s most livable cities according to the Economist Global Liveability Index. Living and working in Copenhagen will be a good experience for you and your family.

Salary and terms of employment
Appointment and salary will be in accordance with the Ministry of Taxation’s agreement with the Danish Confederation of Professional Associations (AC) and job structure for academic staff at Universities.

Qualification Requirements

The following qualifications are required for the PhD position:

- 3 year programme: candidates should have a MSc (or equivalent)
- 4 year programme: candidates should have a BSc and one year of university studies on Master level

The applicants will be employed and enrolled at the ITU University for a period of 3 or 4 years depending on the university degree level of the applicant.

The following qualifications are required for the Postdoc position:

- Candidates are required to have academic qualifications at PhD level

Employment as postdoc is a temporary position for up to 4 years at the same University.

Application
The application and all other documents ought to be in English and must include:

- A motivated application (cover letter), including a detailed description of your qualifications
- A full CV, including name, address, phone number, e-mail, previous and present employment and academic background
- Documentation of academic degrees (copy of degree certificates etc.)
- A statement of purpose, which provides evidence of independent thinking, novelty and originality, of the targeted research field matching the project description
- For candidates applying for the PhD position: relevant scientific publication or/and master thesis written by the applicant
- For candidates applying for the PhD position: documentation of English proficiency
- For candidates applying for the Postdoc position: relevant scientific publication or/and PhD thesis written by the applicant
- For candidates applying for the Postdoc position: documentation of pedagogical experience and teaching record within one or more of the teaching areas listed for the position

Applications without the above-mentioned required documents will not be assessed.

The applicant will be accessed according to the Appointment Order from the Ministry of Science, Technology and Innovation of 13 March 2012.

The IT University may use shortlisting in connection to the recruitment process. In case of shortlisting, the Chair of the hiring committee selects applicants for assessment in consultation with the hiring committee after the application deadline. All applicants are notified whether their application has been passed for assessment. The shortlisting of candidates for assessment is based on the criteria in the job posting.

Further information
Questions about the positions can be directed to project leader and associate professor Veronika Cheplygina, IT University of Copenhagen, vech@itu.dk.
Early contact is strongly encouraged.
Questions related to the application procedure may be directed to HR, hr@itu.dk.

Application procedure
You can only apply for this position through our e-recruitment system. Apply by pressing the button "Apply for position" in the job announcement on our website.

For PhD position applicants, please read the guidelines for applicants carefully before filling in the application form. You can see the guidelines through this link.
The IT University might upon request in the interview process contact 1-2 references.

Please note that all application material will be destroyed after the assessment.

Application deadline: 30 April 2025 at 23:59 CEST.

Applications/enclosures received at ITU after the application deadline will not be taken into consideration. If you submit an application, it is your responsibility to ensure that it arrives before the deadline so please allow sufficient time for upload of publications and other documents.

At The IT University of Copenhagen we have an international environment and view diversity as a valuable resource. We are committed to ensuring an inclusive working environment for all our employees and encourage all qualified applicants, regardless of age, gender, religious affiliation, or ethnic background to apply for the position.