Post-Training a 7B Model for One Enterprise's Code

HardLLMs & GenAI

Reported in interviews at Cohere

Problem statement

How would you adapt and serve a coding model for a customer’s internal codebase? You may train on scrubbed repositories, reviews, test results and opted-in logs, but data and weights must stay in the customer’s cloud tenant. Improve accepted suggestions and completed tasks while preserving general coding ability.

Explain when preferences help, diagnose reward gaming and design training and reward validation.

Show hint

Compare what a model learns from imitating someone else's chosen answer with what it learns from being scored on answers it produced itself.

Relevant roles: ML Engineer, LLM Engineer, Gen AI Engineer and Research Scientist.

Related free problems