TWiT 1065: AI Action Park - DeepSeek's mHC Model Training Breakthrough!...
This Week in Tech (Audio)Full Title
TWiT 1065: AI Action Park - DeepSeek's mHC Model Training Breakthrough!
Summary
The episode discusses the rapid advancements and evolving landscape of AI in 2025 and anticipation for 2026, highlighting the productization of AI by companies like Anthropic and Google, the ongoing debate about AI's impact on programming professions, and the cybersecurity implications of AI.
The hosts also touch upon regulatory challenges and consumer-facing AI developments, as well as the historical parallels of technological disruption.
Key Points
- The AI industry is seeing a significant shift towards productization, with companies like Anthropic and Google focusing on business applications, while consumer AI products from Google and OpenAI are showing rapid improvement and adoption.
- There is a growing sentiment among AI practitioners, like Andrej Karpathy, that the pace of AI development is overwhelming, leading to a feeling of being left behind, which mirrors past technological shifts like the advent of the internet.
- Concerns about the environmental impact of AI, specifically water consumption, are being re-evaluated, with evidence suggesting these concerns may be overblown compared to other industries like golf courses.
- The practice of "hackquisitions," where companies acquire talent by buying other firms without necessarily acquiring the entire company, is a growing trend, exemplified by NVIDIA's deal with Grok.
- OpenAI is actively seeking a Head of Preparedness to manage the risks associated with advanced AI, indicating a focus on safety and mitigation alongside development.
- The increasing capability and accessibility of AI tools are transforming workflows and productivity, drawing parallels to the impact of early software like Photoshop.
- Regulatory efforts are underway to address AI-related issues, but also pose potential threats to internet freedoms, such as age verification requirements and attempts to undermine Section 230.
- The cybersecurity risks associated with AI are significant, ranging from AI-powered phishing attacks to data leakage through AI applications, necessitating robust security solutions like zero trust architectures.
- The semiconductor industry faces supply chain challenges and geopolitical tensions impacting AI development, particularly concerning China's access to advanced chip-making technology.
- Waymo's autonomous vehicles have encountered operational issues during power outages, highlighting the dependence of AI on infrastructure and the need for robust fallback mechanisms.
- The evolution of payment systems for public transportation, like New York's MetroCard phasing out in favor of tap-to-pay, reflects broader technological shifts towards digital and contactless solutions.
- The passing of Stuart Sheffey, a pioneer in computer television shows, is remembered, underscoring the historical roots of tech communication and the evolution of the industry.
Conclusion
The AI landscape is rapidly evolving, with significant progress in productization and integration, but also facing challenges in accessibility, ethics, and societal impact.
Historical parallels of technological disruption provide context for understanding the current AI revolution, suggesting that while overwhelming, these changes can also lead to unprecedented opportunities.
The importance of balancing technological advancement with user safety, ethical considerations, and robust cybersecurity remains paramount as AI becomes more pervasive.
Discussion Topics
- How are AI advancements shaping the future of work for programmers and creative professionals, and what strategies are essential for staying relevant?
- What are the most significant ethical and societal challenges posed by the rapid development of AI, and how can they be addressed effectively?
- With the increasing complexity and interconnectedness of AI, what are the most critical cybersecurity measures individuals and organizations need to adopt to protect themselves from emerging threats?
Key Terms
- TPU
- Tensor Processing Unit, a custom ASIC developed by Google for neural network machine intelligence, specifically for machine learning.
- Vibe Coding
- A term coined by Andrej Karpathy referring to writing code in a more intuitive and fluid way, often facilitated by AI coding assistants.
- Generative AI
- A type of artificial intelligence capable of generating new content, such as text, images, music, or code, often in response to prompts.
- LLM
- Large Language Model, a type of AI model trained on massive amounts of text data, capable of understanding and generating human-like text.
- Hallucination (AI)
- When an AI model generates false or misleading information presented as factual.
- Hackquisition
- A term suggesting the acquisition of a company primarily to hire its talent and intellectual property, rather than the company as a whole.
- Manifold Constrained Hyperconnections (mHC)
- A new AI training technique proposed by DeepSeek aimed at improving training stability, scale, and efficiency.
- Burkhoff Polytope
- A mathematical concept related to matrices and stochastic processes, used in the description of DeepSeek's mHC technique.
- Stochastic
- Involving random variables or processes; having a probability distribution.
- OLAP
- Online Analytical Processing, a category of software technology that enables analysts, managers and executives to access, more rapidly, from complex summarization and comparison of business data.
- Flipper Zero
- A portable, tamper-resistant multi-tool for geeks and hackers, designed for penetration testing and digital/physical interaction with various systems.
- Raspberry Pi
- A series of small single-board computers based on ARM architecture, popular for hobbyist and educational projects.
- CES
- Consumer Electronics Show, an annual trade show organized by the Consumer Technology Association, showcasing new technology and products.
- QD-OLED
- A type of OLED display technology that uses quantum dots to enhance color and brightness.
- DRAM
- Dynamic Random-Access Memory, a type of semiconductor memory that uses capacitors to store each bit of data.
- VRAM
- Video Random-Access Memory, a specialized type of RAM used by graphics cards for storing image data.
- RSU
- Restricted Stock Unit, a company-issued stock grant that is subject to vesting conditions.
- TSMC
- Taiwan Semiconductor Manufacturing Company, the world's largest contract chip manufacturer.
- EUV
- Extreme Ultraviolet Lithography, a technique used in semiconductor manufacturing to etch patterns onto silicon wafers.
- MTA
- Metropolitan Transportation Authority, the public authority responsible for most public transportation in the New York metropolitan area.
- MetroCard
- A contactless fare card used for public transportation in the New York metropolitan area.
- Omni (One Metro New York)
- A new contactless fare payment system for the MTA, replacing the MetroCard.
- Computer Chronicles
- A long-running television show that documented the personal computer industry from 1981 to 2002.
- CPM
- Control Program for Microcomputers, an operating system developed by Digital Research.
- Digital Research
- A software company founded by Gary Kildall.
- HYPERCARD
- A software construction kit, combining a database with an object-oriented programming language and a graphical user interface.
- SUPERCard
- An extension of HyperCard.
- DIRECTOR
- A multimedia authoring tool developed by Macromedia.
- HYPERCARD
- A software construction kit, combining a database with an object-oriented programming language and a graphical user interface.
Timeline
Introduction of Joey Davila, AI developer advocate, and Dan Patterson, Senior Director of Content at Blackbird.ai.
Discussion on the rapid pace of AI development and its potential to change everything.
Analysis of AI's economic impact and the productization trend of AI companies.
Debunking myths about AI's water consumption and discussing the broader environmental context.
Conversation on the progress made in AI, referencing companies like Anthropic and Google.
Discussion on the strategic direction of AI companies, particularly their focus on consumer products.
Comparison of AI's impact on journalism and productivity to the early days of Photoshop.
Discussion of Andrej Karpathy's sentiment about falling behind in the AI programming field.
Reflection on the overwhelming feeling of technological change, drawing parallels to the early internet.
Anecdote about using Mosaic browser in the early days of the internet.
Reflection on the shift from utopian visions of the internet to current cynicism and consequences.
Analogy of YouTube's content explosion to the potential for AI to generate vast amounts of useful or terrible output.
Discussion on the history of domain name ownership and the early internet.
Reflection on the nature of technology podcasting and its slightly OCD audience.
Observation about business acquisitions and moves happening during holiday periods in the AI industry.
News about NVIDIA's acquisition of Grok's technology and talent, termed a "hackquisition."
Discussion of Mark Zuckerberg's acquisition of the Singapore startup Manus.
OpenAI's search for a Head of Preparedness and the associated salary and responsibilities.
Joey Davila's use of Claude Code for job searching and evaluating his fit for roles.
Introduction of DeepSeek's mHC (Manifold Constrained Hyperconnections) model training breakthrough.
Discussion of DeepSeek's analogy for mHC, comparing it to a "crazy water park with perfect safety controls."
Debate between two schools of thought on LLMs: one viewing them as a dead end, the other advocating for more compute.
Reflection on the rapid advancement of AI since ChatGPT's release three years ago.
Discussion of Cory Doctorow's use of Joey's quote and the concept of "shitification."
Personal anecdotes about the impact of SARS and COVID-19 on public health.
Overview of proposed federal "bad internet bills" like the SCREEN Act and the COOPER Davis Act.
Discussion of Things Canary, a security device that acts as a honeypot.
Mention of the FCC banning foreign-made drones and its implications.
Concerns about the potential for new legislation to further attack Section 230.
New York state law requiring social media platforms to display warning labels on content.
A federal judge blocks Texas's App Store age verification law, citing First Amendment concerns.
Discussion of Apple's potential role in age verification for app stores and user privacy.
Recalling early peer-to-peer VPN projects like PeekABoot from Cult of the Dead Cow.
Sharing of past interactions and the history of tech media, including G4 Tech TV.
The hosts look forward to CES coverage next week and mention special content for Club Twit members.
Reflections on the history of tech journalism and the early days of TWiT.
Discussion about US government policies on Chinese chips and the semiconductor industry.
The US FCC bans all foreign-made drones, not just DJI, impacting various industries.
The FCC kills the Cybertrust Mark Program, intended to improve home security device cybersecurity.
Waymo vehicles stopped operating in San Francisco during a power outage, causing traffic disruptions.
Discussion on the impact of ride-sharing services like Uber on New York City's traffic and taxi industry.
San Francisco's new mayor bans Flipper Zeros and Raspberry Pis at the inauguration.
Discussion about the portability and potential misuse of devices like Flipper Zero.
The MetroCard is being phased out in New York City in favor of tap-to-pay systems like Omni.
Apple's supply chain and potential impact of DRAM prices on future MacBook Pro models.
The "dumbest things that happened in tech" in 2025, including a lawyer suing Mark Zuckerberg over his name.
The story of Suhail Doshi exposing an engineer working multiple jobs simultaneously.
Mark Zuckerberg's methods for recruiting talent for OpenAI, including delivering soup.
Elon Musk's Grok AI and its issues with generating inappropriate content, including child nudity.
Discussion on how technology, historically driven by adult content, is evolving.
A humorous tangent about New York pizza and sandwich shops.
Concern that children are losing the ability to read analog clocks due to reliance on digital devices.
Nostalgic discussion about radio show "hot clocks" and the use of cart machines.
Remembering the KFRC mobile studio and the history of radio technology.
The significance of the cart machine in the show "Stranger Things" and its 1980s setting.
Joey Davila's accordion playing and the historical context of accordion music.
Introduction of Redis Cloud as a real-time data platform for AI applications.
Preview of CES 2026, including Sony's Afila car and Samsung's new QD-OLED TVs.
Discussion of the surprising price point and form factor of a new BlackBerry-style Android phone.
The skyrocketing price of DRAM and VRAM, impacting the cost of consumer electronics and AI hardware.
The MTA's transition from MetroCards to tap-to-pay systems, referred to as Omni.
A tribute to Stuart Sheffey, creator of "Computer Chronicles," and the history of early computer television shows.
Joey Davila's ongoing projects, including accordion music and AI development.
Thanks to guests and a look ahead to next week's show covering CES.
A reflection on TWiT's 20-year history and a call to action for listeners to spread the word.
Episode Details
- Podcast
- This Week in Tech (Audio)
- Episode
- TWiT 1065: AI Action Park - DeepSeek's mHC Model Training Breakthrough!
- Official Link
- https://twit.tv/shows/this-week-in-tech
- Published
- January 5, 2026