20VC: Mercor CPO on Revenue Concentration from Frontier Labs...

Full Title

20VC: Mercor CPO on Revenue Concentration from Frontier Labs | Why Large Enterprise is Scared to Partner with Frontier Labs | Why Small Specialised Models is the Future with Osvald Nitski

Summary

This episode features Osvald Nitski, CPO at Mercor, discussing the evolving landscape of AI model development, enterprise adoption, and the future of specialized data providers.

Key themes include the impact of open-source models, enterprise hesitancy with frontier models, the growing importance of specialized AI models, and the challenges and opportunities in the AI data acquisition and deployment market.

Key Points

• Open-source models are seen as raising the floor of AI capabilities, meaning companies will no longer pay for what already exists in basic open models; the focus shifts to unique needs and frontier model performance.
• Enterprise adoption of frontier AI models is hindered by skepticism and fear of sharing sensitive, core business data, leading to a preference for proprietary models for less sensitive, general workflows like HR or procurement.
• The 90/10 split, suggesting 90% of enterprise workflows can be handled by open models, is disputed; Nitski posits a closer to 50% for long-horizon tasks, with other workflows being efficiency-based or requiring continuous improvement rather than a binary "can do" assessment.
• Specialized models tailored to individual company goals (growth, margin, work-life balance) are seen as the future, with Mercor believing these will require enterprise-specific evaluation and training data.
• Despite rapid AI advancement, Nitski believes there isn't currently an "enterprise ROI problem" because the field is in an exploration phase, with more tolerance for experimentation before strict ROI calculations are demanded.
• The rise of services in AI enterprise deployment is viewed as a short-term necessity due to a current lack of AI deployment knowledge across industries, which will eventually be absorbed into job functions as products become more sophisticated.
• For product management in the AI era, the focus is shifting from tool proficiency to business impact, judgment, and understanding user needs, as AI agents handle more of the execution.
• Mercor's success in scaling its marketplace for human data talent is attributed to a strong expert experience (timely, fair pay, dignified work), a robust sourcing team, and fostering a referral program.
• The increasing complexity and demand for "environments" (simulations of apps for agent training) and physical data (robotics) are identified as significant future growth areas in the AI data market.
• The talent war in San Francisco is described as brutal, with a strong bias towards hiring senior candidates who demonstrate agency, ownership, and a "give a shit" attitude, as these traits are harder to coach than technical skills.

Conclusion

The rapid advancement of AI is creating a demand for specialized data and models that can address unique enterprise needs beyond generic capabilities.

Enterprise adoption of cutting-edge AI is cautious due to data sensitivity, highlighting the ongoing need for solutions that balance innovation with security and trust.

The future of AI deployment hinges on democratizing access to complex AI workflows and a continued focus on human judgment and strategic decision-making alongside AI capabilities.

Discussion Topics

• How can enterprises overcome their skepticism and fear of sharing sensitive data to fully leverage the benefits of frontier AI models?
• What are the most promising future applications for specialized AI models and data in industries beyond general AI, such as robotics or advanced cybersecurity?
• As AI agents become more integrated into workflows, how can individuals and teams maintain their critical thinking, creativity, and essential skills in the face of increasing automation?

Key Terms

Frontier Models

State-of-the-art AI models that represent the cutting edge of performance and capabilities.

Open-Source Models

AI models whose source code and architecture are publicly available, allowing for widespread use, modification, and distribution.

Synthetic Data

Data that is artificially generated by algorithms, often used to augment or replace real-world data for training AI models.

InstructGPT

An early large language model developed by OpenAI, notable for its ability to follow instructions.

Supervised Fine-Tuning (SFT)

A method of training AI models where they are further trained on labeled datasets to improve performance on specific tasks.

Preference Ranking

A technique used in AI training where human evaluators rank different outputs of a model, guiding it towards preferred responses.

RL Environments

Refers to Reinforcement Learning environments, simulated scenarios where AI agents learn through trial and error by receiving rewards or penalties.

Proprietary Models

AI models that are owned and controlled by a specific company, with their internal workings not publicly disclosed.

Prompt Engineering

The practice of designing and refining inputs (prompts) given to AI models to elicit desired outputs.

Product Management (PM)

The organizational function within a company responsible for the strategy, roadmap, and feature definition of a product.

Engineering (ENG)

The field of applying scientific and mathematical principles to design, build, and maintain structures, machines, systems, and processes.

Agency

In a business context, the ability of individuals or teams to act independently and make their own free choices.

Ownership

In a business context, the act of taking responsibility for a task or project from start to finish.

Timeline