Postdoc in Multimodal Generative Artificial Intelligence

BRAIn (Better Representations for Artificial Intelligence) is a research team within the Lab-STICC (CNRS UMR 6285), part of the Mathematical and Electrical Engineering (MEE) department at IMT Atlantique. The team's mission is to investigate key questions at the intersection of Artificial Intelligence, Deep Learning, and Signal Processing, with applications spanning images, sounds, texts, and more complex domains, including neuroimaging data.

Achieving an 80% reduction in time for corporate video editing is the ambitious goal of the MOVIOLIA project, which brings together EKTACOM, tydeo, and IMT Atlantique. Led by EKTACOM, the BRAIn team contributes its expertise in Artificial Intelligence to MOVIOLIA. To achieve its objectives, the project will leverage foundation models in video, image, audio, and Large Language Models (LLMs) to automate a significant portion of the video editing process.

Despite its concrete application in the corporate context, MOVIOLIA will also tackle fundamental research questions in Generative AI, including:

• Specialization and optimization of foundation models through distillation
• Complexity of Generative AI (GAI) models in terms of computation and parameter count
• Adaptation/fine-tuning of multimodal foundation models

• MISSIONS

Under the supervision of Nicolas Farrugia, and in collaboration with other members of the BRAIn team and the MOVIOLIA project, the postdoctoral researcher will be entrusted with the following missions:

1. 1. Contribute to fundamental research in Deep Learning and multimodal Generative AI (GAI), with applications in the aforementioned fields and in alignment with MOVIOLIA's objectives.
2. 2. Participate in supervising and monitoring the work of a junior engineer recruited as part of the MOVIOLIA project.
3. 3. Support the technology and literature monitoring activities for MOVIOLIA.

Job description for more information

Minimum education and/or experience required:

PhD obtained within the last three years prior to the hiring date in one of the following fields:

• Machine Learning
• Artificial Intelligence
• Video Processing
• Image Processing
• Audio Processing
• Natural Language Processing
• Speech Processing

Essential skills, knowledge and experience:

• Expertise in Deep Learning, preferably with PyTorch, demonstrated through publications accompanied by publicly available code repositories.
• Experience in at least one application domain: audio, speech, video, image, or language.

Desirable skills, knowledge and experience:

• Experience in audio signal processing is highly valued.
• Knowledge of complexity reduction techniques in Deep Learning, such as quantization, distillation, factorization, parameter reduction, etc.
• Experience with fine-tuning and adaptation techniques, such as LoRA/DoRA or similar.
• Experience in leveraging foundation models in the aforementioned application domains.
• Familiarity with ML/Ops methods, including tools such as Docker, Singularity, Kubernetes.

Experience with fast inference frameworks, such as Accelerate, vLLM, TensorRT-LLM.

Abilities and skills:

• Ability to work independently.
• Strong writing skills.

Experience in mentoring interns or junior engineers, with a passion for knowledge sharing.

• OTHER INFORMATION

This project is conducted in partnership with two companies that are experts in video processing. These collaborations aim to combine academic research with industrial expertise to address real-world challenges in video editing and processing, leveraging cutting-edge Artificial Intelligence technologies.

• APPLICATION DOCUMENT
• Curriculum Vitae (CV): Including a comprehensive list of peer-reviewed publications, particularly in journals, international, and national conferences.
• Motivation Letter.
• Thesis Reviewers' Reports.
• Thesis Defense Report.
• GitHub/GitLab or equivalent repositories: Demonstrating experience with open-source coding (highly recommended).

Contact Information for Two Referees: Names and emails of two individuals who can provide a reference for the candidate.