The multi-agent cultural evolution software that we use is available at https://gitlab.inria.fr/moex/lazylav and experiment are stored in our repository https://sake.re.
We have pursued our effort to document the repurposing of our experiments as Jupyter notebooks (https://sake.re). Some neglected benefits of semantically describing experiments provide more arguments in favour of scientific knowledge graphs [Euzenat 2022a]. Beyond being searchable through flat metadata, a knowledge graph of experiment descriptions may be able to provide answers to scientific and methodological questions. This includes identifying non experimented conditions or retrieving specific techniques used in experiments. In turn, this is useful for researchers as this information can be used for repurposing experiments, checking claimed results or performing meta-analyses.
Open science has broadened access to scientific datasets. However, identifying relevant ones to specific user needs remains challenging due to its volume, diversity and poor metadata. We proposed to combine semantically enriched open dataset metadata with LLM-based agents that interpret natural language queries to manage the gap between users’ needs and dataset descriptions, and to support the retrieval of relevant datasets [Dupuis 2025b]. This enables the extraction and refinement of user needs, as well as the generation of justifications for the retrieved results. To assess the performance of the proposed system, an evaluation was conducted across multiple earth observation data request scenarios [Dupuis 2025a].
Reproducibility is a desirable property of scientific research. On the one hand, it increases confidence in results. On the other hand, reproducible results can be extended on a solid basis. In rapidly developing fields such as machine learning, the latter is particularly important to ensure the reliability of research. We contributed to the methodology of reproducing experimental results in the field of machine learning by providing a systematic approach to reproducing (using the available implementation), replicating (using an alternative implementation) and reevaluating (using different datasets) state-of-the-art experiments [Werner 2024a]. This approach enables the early detection and correction of deficiencies and thus the development of more robust and transparent machine learning methods. This work was illustrated by the independent reproduction, replication, and reevaluation of initially published experiments with a method that we wanted to extend. For each step, we identified issues and draw lessons learned and discussed solutions that have proven effective in overcoming the encountered problems. This work can serve as a guide for further reproducibility studies and generally improve reproducibility in machine learning.
| Publications on open science and reproducibility |