EP 47 — Solving the AI Curation Problem with Andrew Hill from...
The DCo PodcastFull Title
EP 47 — Solving the AI Curation Problem with Andrew Hill from Recall
Summary
The episode discusses the need for an AI curation and orchestration layer, introducing Recall as a decentralized skill marketplace.
Recall aims to solve the problem of identifying and utilizing capable AI agents by creating competitive arenas and scoring mechanisms, similar to a "page rank" for AI.
Key Points
- The emergence of LLMs has led to a proliferation of AI agents, creating a need for tools to assess their capabilities and distinguish genuine agents from hype.
- Recall functions as a decentralized skill marketplace where agents compete in time-bounded competitions, aiming for the highest risk-adjusted profit to prove their skill.
- The platform utilizes on-chain competitions and verified results to score agents, allowing organizations to find and fund the AI solutions they need.
- This system is analogous to Google's PageRank, which initially ranked web pages by backlinks and later incorporated user interaction to refine rankings, suggesting a future where user feedback and real-world performance improve AI agent rankings.
- The demand for AI agents is growing, particularly within organizations and decentralized protocols, as AI becomes more capable and cost-effective to develop.
- Google's recent advancements in agent tooling and payments (e.g., AP2, X402) suggest a broader industry trend towards supporting and decentralizing AI agent development.
- The discussion touches upon the risks of AI models gaming systems and the necessity of verification and measurement tools to ensure alignment with desired outcomes.
- Recall categorizes its users into four groups: agent builders, curators, boosters, and skill pool funders, each contributing to the platform's ecosystem.
- The platform is developing mechanisms for distilling human judgment and using AI judges to evaluate agent performance, especially for more subjective tasks.
- The long-term vision for Recall is to become a neutral evaluation layer for AI quality and alignment, potentially scaling to evaluate and align advanced AI like AGI or ASI.
Conclusion
The increasing sophistication and accessibility of AI agents necessitate robust curation and verification platforms like Recall to ensure discoverability and alignment.
The future of AI involves decentralized skill marketplaces where agents compete to solve specific problems, driving innovation and value.
Recall aims to be a neutral, open layer for AI evaluation and alignment, fostering a competitive ecosystem that benefits both AI developers and users.
Discussion Topics
- How can the success of AI agents in competitive arenas translate into real-world problem-solving for businesses?
- What are the potential ethical implications of a decentralized AI skill marketplace and how can they be mitigated?
- As AI agents become more autonomous, what role should human oversight play in their development and deployment?
Key Terms
- LLM
- Large Language Model - A type of artificial intelligence model trained on vast amounts of text data to understand and generate human-like language.
- Risk-adjusted profit
- A measure of profit that takes into account the amount of risk taken to achieve it, aiming for higher profits with lower risk.
- PageRank
- An algorithm used by Google Search to rank web pages in their search engine results, based on the number and quality of backlinks.
- Decentralized skill marketplace
- A platform where skills or services are offered and traded without a central authority, often leveraging blockchain technology.
- Agent (AI)
- A software entity that acts autonomously to achieve goals, often using AI and machine learning.
- AGI
- Artificial General Intelligence - AI that possesses the ability to understand, learn, and apply knowledge across a wide range of tasks at a human level.
- ASI
- Artificial Superintelligence - AI that surpasses human intelligence and ability across virtually all fields.
- RL
- Reinforcement Learning - A type of machine learning where an agent learns to make decisions by performing actions in an environment to maximize a reward signal.
- Open-weight models
- AI models whose architecture and parameters are publicly available, allowing for greater transparency and modification.
- Compute
- The processing power required to run AI models, often involving significant hardware and energy resources.
Timeline
Recall creates an arena where agents compete for risk-adjusted profit to identify consistently capable agents.
The realization of the need for AI curation stemmed from observing the hype and lack of clarity around AI agents post-LLM boom.
Recall builds a decentralized skill marketplace where organizations can fund needed AI solutions, with agents competing to demonstrate capability.
Real-world competitions on Recall verify agent results and score them, creating a discoverable ranking of agent performance.
Trading competitions are a concrete example of Recall's skill pools, where agents compete for risk-adjusted profit in areas like perpetual futures.
External blockchain projects can create skill markets on Recall to identify capable agents for their protocols.
The future vision includes agents building the Recall protocol itself and skill markets becoming economic drivers for valuable AI work.
The evolution of PageRank from backlinks to incorporating click-through rates provides a model for how AI agent ranking can become more sophisticated.
The challenge of bootstrapping both supply and demand for new skills is addressed by the existing demand from organizations and the abundance of agent builders.
The proliferation of AI tooling, including from OpenAI and open-source projects, is expected to accelerate agent development.
Google's support for agent payment rails (AP2, X402) validates the trend towards decentralized agent efforts.
The agentic phase of AI, with capabilities like planning and tool use, is now mature, leading to the practical application of AI agents.
The decreasing cost of software development, driven by AI, is lowering the barrier to entry for building specialized agents.
The reduction in software development costs allows for the creation of niche and personalized AI solutions that were previously too expensive.
Measuring performance is easier in trading due to objective data, but challenges arise with more subjective skills.
Recall is building blocks for verification, including trading competitions and prediction markets, and is exploring distilling human judgment.
Subjective skill evaluation can be addressed through distilling human judgment, crowd-sourced pairwise assessments, or AI-driven judges.
AI agents often try to "game the system" to reach their goals, highlighting the need for verification tools.
The current RL models are in their first generation, focusing on pre-training, but post-training and online learning are emerging.
Key user personas on Recall include agent builders, curators, boosters, and skill pool funders.
Revenue lines for Recall include agents competing with stakes, transactions within skill markets, and boosting games.
The boost mechanism in Recall is for identifying capable curators and forecasting agent performance, not for directly impacting agent rankings.
Agents may eventually stake their own capital to compete in high-value markets, and skill markets themselves will generate transactions.
Human intervention in judging is considered for ambiguous cases where automated judges lack consensus.
The decrease in software development costs allows teams to focus on niche AI demands and build specialized models or agents.
Recall is moving into less complex skills as demand is unlocked, expanding beyond challenging full-portfolio management.
Potential risks for Recall include the uncertainty of distributed intelligence versus centralized AI platforms and the immense resources (compute, energy, capital) required for AI development.
The model envisions niche agents for specific skills, requiring complex tasks to be handled by multiple agents working together.
The platform has seen significant user engagement with over 1.5 million people curating agents, and encourages participation and experimentation.
The rapid evolution of AI technology, including open-weight models and post-training fine-tuning, creates opportunities for continuous improvement and new AI applications.
Episode Details
- Podcast
- The DCo Podcast
- Episode
- EP 47 — Solving the AI Curation Problem with Andrew Hill from Recall
- Official Link
- https://www.decentralised.co/podcast
- Published
- October 28, 2025