What is Goldener? | Why open research? | What organisation? | Themes | General resources

Goldener is an open-source Python library (Apache 2 licence) for orchestrating data across the full lifecycle of Artificial Intelligence (AI) pipelines, from the training phase to the monitoring phase. Goldener is designed to make the right data available at the right time, helping teams optimize AI performance while reducing costs in annotation, training, evaluation, and monitoring.

All of Goldener's data centric AI features are built from the same core principle: the semantics of data can be described by embeddings extracted from pre-trained/foundational models. This semantic representation is general enough to emphasize the differences or similarities between data samples, and it is what every topic in this repository ultimately builds on.

This open research repository aims to centralize information helpful for driving Goldener: bibliography, promising directions, research plans, experiment results, ...

Goldener is open source. It evolves mostly through the commitment of its community. Goldener's contributors are all convinced the standard practices within the Artificial Intelligence (AI) lifecycle are suffering from inefficiencies and inadequacies. These convictions come either from proven demonstrations, whether public or not, or from assumptions driven by their skills and experiences. Our mutual goal is to improve efficiency and reliability throughout the full AI lifecycle.

Goldener's target is broad (full AI lifecycle, data- and task-agnostic, customizable), and the feature possibilities are numerous throughout the AI lifecycle. In addition, Goldener is a combination of the infrastructure for orchestrating data and the algorithms that fix the inefficiencies. Thus, its development requires a collaborative approach between developers making it production ready and researchers designing algorithms.

By sharing knowledge and results in this repository, our community aims to:

• Inspire researchers: Some information might lead people to reflect on a topic and design their own research plan about it. Hopefully, heading to peer-reviewed publications.
• Encourage contribution: Some content might be a starting point for useful new features, leading contributors to propose and implement them.
• Build practitioner trust: Openly available results demonstrate the effectiveness and reliability of Goldener's features, ensuring the soundness of our claims.
• Foster collaboration: The community can benefit from each other's insights and experiences.
• Empower AI agents: The coding/research agents require context to be efficient; this source of information can lead them to better decisions while running their tasks to contribute to Goldener.

This repository is meant to gather research material and results that can be useful to the community. It is organized around topics/themes, each with a dedicated folder that might include:

• Resources: The most important papers, researchers, and tools contributing to the theme. Researchers can quickly identify relevant prior work, compare methods, and build on established foundations. In addition, practitioners can find relevant demonstrations of Goldener's added value. Finally, it can become as well a useful inspiration for new Goldener features.

• Ideas/Questions: A collection of research ideas/questions shared by the community. This is like a continuous written brainstorming session that can inspire new studies and features.

• Plans: Concrete research plans turning ideas into actionable experiments using Goldener. For instance, it can correspond to experiments already run by the Goldener community, wider project description for research funding, or proposals for student projects.

• Results: Documented outcomes of experiments and studies conducted using Goldener. This section allows the community to review findings, validate approaches, and build on previous work.

• Augmentation: Selection of relevant augmentation for the training in AI pipelines.
  
• Batching: Optimizing batch creation during the training in AI pipelines.
  
• Data selection: Selection of relevant data for training and monitoring in AI pipelines.
  
• Drift: Detection of drift in data (covariate drift, concept drift, and label drift).
  
• Frameworks: Various frameworks available for AI lifecycle optimization.
  
• Labeling: Efficient labeling optimizing the training and monitoring in AI pipelines.
  
• Losses: Implementation of loss functions optimizing the training in AI pipelines.
  
• Model design: Selection of the best models and hyperparameters in AI pipelines.
  
• Out of distribution: Detection of out-of-distribution data in AI pipelines.
  
• Training strategy: Selection of the best training strategy in AI pipelines.

The general resources gather the material that serves Goldener's goal across several themes at once: data-centric AI as a whole, the semantic representation of data through embeddings from pretrained/foundation models (Goldener's core principle), and studies of how the data shapes the full AI lifecycle (e.g. the impact of randomness in data-related choices during training). Material that clearly belongs to a single theme goes into that theme's folder instead, and work that does not use the data itself to improve training or monitoring is out of scope.

📚 Bibliography: cross-cutting papers and references on data-centric AI
💡 Ideas: open questions and research directions spanning several themes
👥 Researchers: authors and contributors whose work spans several themes