Applications of Robotics

As we transition from traditional task-based automation to 𝗮𝘂𝘁𝗼𝗻𝗼𝗺𝗼𝘂𝘀 𝗔𝗜 𝗮𝗴𝗲𝗻𝘁𝘀, understanding 𝘩𝘰𝘸 an agent cognitively processes its environment is no longer optional — it's strategic. This diagram distills the mental model that underpins every intelligent agent architecture — from LangGraph and CrewAI to RAG-based systems and autonomous multi-agent orchestration. The Workflow at a Glance 1. 𝗣𝗲𝗿𝗰𝗲𝗽𝘁𝗶𝗼𝗻 – The agent observes its environment using sensors or inputs (text, APIs, context, tools). 2. 𝗕𝗿𝗮𝗶𝗻 (𝗥𝗲𝗮𝘀𝗼𝗻𝗶𝗻𝗴 𝗘𝗻𝗴𝗶𝗻𝗲) – It processes observations via a core LLM, enhanced with memory, planning, and retrieval components. 3. 𝗔𝗰𝘁𝗶𝗼𝗻 – It executes a task, invokes a tool, or responds — influencing the environment. 4. 𝗟𝗲𝗮𝗿𝗻𝗶𝗻𝗴 (Implicit or Explicit) – Feedback is integrated to improve future decisions.     This feedback loop mirrors principles from: • The 𝗢𝗢𝗗𝗔 𝗹𝗼𝗼𝗽 (Observe–Orient–Decide–Act) • 𝗖𝗼𝗴𝗻𝗶𝘁𝗶𝘃𝗲 𝗮𝗿𝗰𝗵𝗶𝘁𝗲𝗰𝘁𝘂𝗿𝗲𝘀 used in robotics and AI • 𝗚𝗼𝗮𝗹-𝗰𝗼𝗻𝗱𝗶𝘁𝗶𝗼𝗻𝗲𝗱 𝗿𝗲𝗮𝘀𝗼𝗻𝗶𝗻𝗴 in agent frameworks Most AI applications today are still “reactive.” But agentic AI — autonomous systems that operate continuously and adaptively — requires: • A 𝗰𝗼𝗴𝗻𝗶𝘁𝗶𝘃𝗲 𝗹𝗼𝗼𝗽 for decision-making • Persistent 𝗺𝗲𝗺𝗼𝗿𝘆 and contextual awareness • Tool-use and reasoning across multiple steps • 𝗣𝗹𝗮𝗻𝗻𝗶𝗻𝗴 for dynamic goal completion • The ability to 𝗹𝗲𝗮𝗿𝗻 from experience and feedback    This model helps developers, researchers, and architects 𝗿𝗲𝗮𝘀𝗼𝗻 𝗰𝗹𝗲𝗮𝗿𝗹𝘆 𝗮𝗯𝗼𝘂𝘁 𝘄𝗵𝗲𝗿𝗲 𝘁𝗼 𝗲𝗺𝗯𝗲𝗱 𝗶𝗻𝘁𝗲𝗹𝗹𝗶𝗴𝗲𝗻𝗰𝗲 — and where things tend to break. Whether you’re building agentic workflows, orchestrating LLM-powered systems, or designing AI-native applications — I hope this framework adds value to your thinking. Let’s elevate the conversation around how AI systems 𝘳𝘦𝘢𝘴𝘰𝘯. Curious to hear how you're modeling cognition in your systems.

Exciting updates on Project GR00T! We discover a systematic way to scale up robot data, tackling the most painful pain point in robotics. The idea is simple: human collects demonstration on a real robot, and we multiply that data 1000x or more in simulation. Let’s break it down: 1. We use Apple Vision Pro (yes!!) to give the human operator first person control of the humanoid. Vision Pro parses human hand pose and retargets the motion to the robot hand, all in real time. From the human’s point of view, they are immersed in another body like the Avatar. Teleoperation is slow and time-consuming, but we can afford to collect a small amount of data.  2. We use RoboCasa, a generative simulation framework, to multiply the demonstration data by varying the visual appearance and layout of the environment. In Jensen’s keynote video below, the humanoid is now placing the cup in hundreds of kitchens with a huge diversity of textures, furniture, and object placement. We only have 1 physical kitchen at the GEAR Lab in NVIDIA HQ, but we can conjure up infinite ones in simulation. 3. Finally, we apply MimicGen, a technique to multiply the above data even more by varying the *motion* of the robot. MimicGen generates vast number of new action trajectories based on the original human data, and filters out failed ones (e.g. those that drop the cup) to form a much larger dataset. To sum up, given 1 human trajectory with Vision Pro  -> RoboCasa produces N (varying visuals)  -> MimicGen further augments to NxM (varying motions). This is the way to trade compute for expensive human data by GPU-accelerated simulation. A while ago, I mentioned that teleoperation is fundamentally not scalable, because we are always limited by 24 hrs/robot/day in the world of atoms. Our new GR00T synthetic data pipeline breaks this barrier in the world of bits. Scaling has been so much fun for LLMs, and it's finally our turn to have fun in robotics! We are creating tools to enable everyone in the ecosystem to scale up with us: - RoboCasa: our generative simulation framework (Yuke Zhu). It's fully open-source! Here you go: http://robocasa.ai - MimicGen: our generative action framework (Ajay Mandlekar). The code is open-source for robot arms, but we will have another version for humanoid and 5-finger hands: https://lnkd.in/gsRArQXy - We are building a state-of-the-art Apple Vision Pro -> humanoid robot "Avatar" stack. Xiaolong Wang group’s open-source libraries laid the foundation: https://lnkd.in/gUYye7yt - Watch Jensen's keynote yesterday. He cannot hide his excitement about Project GR00T and robot foundation models! https://lnkd.in/g3hZteCG Finally, GEAR lab is hiring! We want the best roboticists in the world to join us on this moon-landing mission to solve physical AGI: https://lnkd.in/gTancpNK

Presenting FEELTHEFORCE (FTF): a robot learning system that models human tactile behavior to learn force-sensitive manipulation. Using a tactile glove to measure contact forces and a vision-based model to estimate hand pose, they train a closed-loop policy that continuously predicts the forces needed for manipulation. This policy is re-targeted to a Franka Panda robot with tactile gripper sensors using shared visual and action representa- tions. At execution, a PD controller modulates gripper closure to track predicted forces -enabling precise, force-aware control. This approach grounds robust low- level force control in scalable human supervision, achieving a 77% success rate across 5 force-sensitive manipulation tasks. #research: https://lnkd.in/dXxX7Enw #github: https://lnkd.in/dQVuYTDJ #authors: Ademi Adeniji, Zhuoran (Jolia) Chen, Vincent Liu, Venkatesh Pattabiraman, Raunaq Bhirangi, Pieter Abbeel, Lerrel Pinto, Siddhant Haldar New York University, University of California, Berkeley, NYU Shanghai Controlling fine-grained forces during manipulation remains a core challenge in robotics. While robot policies learned from robot-collected data or simulation show promise, they struggle to generalize across the diverse range of real-world interactions. Learning directly from humans offers a scalable solution, enabling demonstrators to perform skills in their natural embodiment and in everyday environments. However, visual demonstrations alone lack the information needed to infer precise contact forces.

I built a self-hosted AI architecture that runs without internet, no API cost, no cloud. AI works when network doesn't. This was the toughest project I’ve ever worked on and I did it to answer one question: Can we talk to AI when the internet is down and can we trust AI with sensitive data which cannot leave the building? Short answer: Yes. Meet Secure AI Lab. What it does: Works like ChatGPT, but lives on your computer and runs without internet Reads your own documents (protocols/policies) to answer with context. Automates tasks (save files, generate PDFs, log entries) locally. Runs fully offline after setup no cloud, no API keys, no telemetry. In the video, I switch Wi-Fi OFF and ask: “What medications are used for cardiac arrest??” OpenWebUI (local chatbot) answers from my local knowledgebase. n8n (local workflow) auto-creates a file on my disk with the summary. Every step happens on localhost. Nothing leaves the machine. ⚠️ Demo ≠ diagnosis. The medication shown is mock data; this is a clinical support example, not medical advice. Why this matters: Emergency Departments (ED) during downtime: keep triage guidance, protocol recall, and order prep running when EHR/internet is down. Hospitals, banks, factories: when privacy and reliability matter, local beats cloud. Cost control: one-time setup vs. indefinite per-token bills. How it works (simple flow) Inside the Lab: Local Brain – AI model (Ollama) generates answers on device. Your Documents – RAG reads your PDFs (protocols/policies) locally. Local Robot – n8n automations save files, generate PDFs, log to SQLite, print if needed. Not just Ollama offline. I built a complete offline system: chat UI + local RAG over my PDFs + automations that create PDFs/logs on disk, with Wi-Fi OFF and no egress. It’s a product, not just a model. I have added File-Watcher: when OpenWebUI saves a new answer, n8n auto-detects it and creates a PDF/log instantly, still with no internet. Stack at a glance OpenWebUI – local chat UI + RAG Ollama – runs the AI models on device n8n – no-code automations (write files, PDFs, logs) Docker – isolated, reproducible setup RAG – reads your docs; answers with citations SQLite/Files – local logs & artifacts (no cloud) This was my toughest build yet. I spent many weeks planning and stitching everything together to prove AI can run fully offline and still be useful in emergencies.

The global economy has an impending problem. While AI is compounding its ability at a historic rate, an aging population and declining fertility rates are already causing labor shortages. These trends, combined with declining costs of robotics hardware, underpin a compelling case for humanoid robots and physical AI. According to Morgan Stanley, the humanoid robot market is set to exceed $5 trillion by 2050. Even in 2025, the larger robotics space saw $21 billion of VC capital invested. And with a steady increase in patent activity mentioning “humanoid” over the past few years, these machines are already walking onto factory floors. For most of human history, productive output was a function of human muscle. Agriculture, manufacturing, logistics, and construction were all built around the physical limits of the human body. Because humans did the work, the built world standardized around human form: doorways, staircases, countertops, and tools are all designed for two arms, two legs, and hands that grip. Redesigning every factory, warehouse, and home around task-specific machines would be unfeasible. A humanoid robot that fits into existing infrastructure doesn’t need the world to change around it. Near-term use cases focus on structured, predictable settings, enabling a robot to learn quickly, make mistakes cheaply, and improve rapidly. My research team at Social Capital concluded that humanoid Robots will have the highest impact in these 7 areas: 1. Domestic Assistance: Supporting mobility needs, handling household chores, and providing medication reminders. 2. Manufacturing: Assisting assembly tasks, moving tools and parts, inspecting finished products. 3. Security & Monitoring: Patrolling facilities, investigating alerts, and assisting in emergencies. 4. Customer Service & Reception: Greeting and directing visitors, answering questions, and managing check-ins or bookings. 5. Facility Maintenance: Conducting routine inspections, performing minor repairs, cleaning, and sanitizing spaces. 6. Healthcare: Assisting nurses, delivering supplies or meals, monitoring patients. 7. Warehouse and Logistics: Picking and packing items, loading and unloading goods, and moving inventory in warehouses. By 2050, Morgan Stanley estimates that more than 1 billion humanoid robots could be working globally, with a market size of over $5 trillion. This is one of the biggest opportunities in the AI era.

As we return to work after the holiday break with energy and renewed focus, I want to challenge you to think deeply in the coming year about how you can use your unique skills to solve problems with the greatest impact. Start with those closest to you, the people in your community. Consider loneliness. It has become a public health crisis, affecting one in six people worldwide with numbers increasing significantly with age. The technology designed to connect us, such as social media, smarter phones, has made many feel more isolated. But if technology is a cause, it can also be a solution. Companion robots have shown real promise in addressing this crisis. In Canada, long-term care facilities and hospitals have adopted robots like Pepper, Paro, and Lovot to support mental health and wellbeing. A study of Paro found 95% of dementia patients showed measurable improvements in mood, reduced agitation, and better sleep. We are still early days, but what’s clear is that the solution requires us to think deeply about how to use technology with purpose. Not to think in black and white. As technologists, it is our duty to solve hard human problems with technology, not create them. For more, read my full tech predictions for 2026 and beyond here https://lnkd.in/eHjPnxYB Now Go Build

Humanoid robots are making robotics visible again. But the real challenge is not simply building a robot that can walk, lift, or manipulate objects. The real challenge is building the full stack around it. Any robot operating in the real world depends on far more than one impressive subsystem: • Sensors and compute • Embedded software • Perception and AI models • Planning and control • Safety systems • Cloud connectivity • Fleet operations • Cybersecurity • Data pipelines • Integration with customer infrastructure This is where robotics becomes difficult. A humanoid demo may show capability. But a production robot must show reliability, safety, maintainability, and economic value, day after day, in messy real-world environments. That requires deep integration across hardware, software, AI, cloud, safety, and operations. In my view, the real moat in robotics will not be one component. It will be integration complexity. The companies that scale robotics successfully will be those that can turn many complex subsystems into one reliable product experience. This also changes how robotics teams need to be built. The strongest robotics organizations will not look like pure hardware teams or pure AI teams. They will look like full-stack systems organizations, combining AI/ML, embedded software, controls, cloud platforms, safety engineering, cybersecurity, product integration, and field operations. Humanoids may be the visible symbol of the next robotics wave. But the real winner will be the team that can integrate the full stack well enough to make robots reliable, safe, and useful in the real world. Curious how others see this: Is the next robotics moat hardware, AI, or full-stack integration? #Robotics #AI #Humanoids #AutonomousSystems #IndustrialAI #SystemsEngineering

The Humanoid Robot Revolution: Timelines, Leadership & Business Impact (Mid-2026 Update) Humanoid robotics has shifted from prototypes to real-world deployment faster than most expected. This is one of the most transformative technological accelerations of our time. Current Leaders Delivering Results: - Figure AI robots are already working in BMW plants on complex tasks. - Tesla is scaling Optimus production, targeting broader availability in late 2026–2027. - Chinese companies (Unitree, Agibot, UBTECH) have shipped thousands of units, leading in volume, cost, and early profitability. - 1X, Agility Robotics (Digit), Boston Dynamics (Atlas) and others are advancing pilots into enterprise and early consumer use. What to Expect: - 2026–2027: Industrial deployments scale across manufacturing, logistics, and warehousing. Early home/service models emerge. Prices drop rapidly. - 2028–2030: Tens to hundreds of thousands deployed globally. Humanoids become reliable 24/7 partners for repetitive and hazardous work. - 2030+: Mass adoption with millions of units, creating a new layer of physical AI that augments human capabilities at scale. Driven by AI breakthroughs, real-world data, and hardware iteration, years of projected progress have been compressed into months. How This Changes Everything Humanoids address labor shortages, aging populations, and human physical limits. They enable 24/7 operations, supply chain resilience, manufacturing reshoring, and lower costs, driving higher productivity and economic abundance. Who Will Lead? A global race with multiple winners: - Tesla: Scale, vertical integration, and AI-data flywheel. - U.S. innovators (e.g. Figure AI): Sophisticated enterprise AI systems. - Chinese manufacturers: Speed, cost leadership, and volume. - Specialists (Boston Dynamics, Agility, 1X): Agility and targeted applications. Impact on Business & Jobs Businesses: Early adopters gain major advantages in cost, output, flexibility, and resilience. Robot-as-a-Service models are emerging. Late adopters will struggle to compete. Jobs: Not replacement, but transformation. Routine, dangerous, and physically demanding roles will be automated, while new opportunities grow in robot maintenance, programming, integration, and entirely new fields. Technology has always created more value and jobs than it displaces. This is a story of amplification: human ingenuity + intelligent machines = safer work, better services, and more space for creativity, strategy, and meaningful contributions. The acceleration is here. The window to prepare is now. What timeline do you see for humanoid adoption in your industry? How is your organization getting ready?

First fully open Action Reasoning Model (ARM); can ‘think’ in 3D & turn your instructions into real-world actions: [📍 Bookmark for later] A model that reasons in space, time, and motion. It breaks down your command into three steps: ✅ Grounds the scene with depth-aware perception tokens ✅ Plans the motion through visual reasoning traces ✅ Executes low-level commands for real hardware Think of it as chain-of-thought for physical action. Give it an instruction like “Pick up the trash” and MolmoAct will: 1. Understand the environment through depth perception 2. Visually plan the sequence of moves 3. Carry them out… while letting you see the plan overlaid on camera frames before anything moves It’s steerable in real time: draw a path, change the prompt, and the trajectory updates instantly. AAAANNNDDD: It’s completely open: checkpoints, code, and evaluation scripts are ALL PUBLIC! Resources Models: https://lnkd.in/dcMVV29k Data: https://lnkd.in/dUwszSvd 📍Blog: https://lnkd.in/diNJFXEi MolmoAct runs across different robot types (from gripper arms to humanoids) and adapts quickly to new tasks. It outperforms models from major labs like NVIDIA, Google, and Microsoft on benchmark tests for generalization and real-world success rates. For anyone building robotics systems or studying AI-driven action models, this is worth exploring… and worth sharing! ♻️

Production changes everything. What worked in a demo starts breaking at scale. That’s where real AI systems are tested. Here are the concepts that actually matter 👇 - Prototype vs production A demo works in controlled conditions, while production systems deal with scale, failures, and messy edge cases. - Training vs inference Training happens occasionally to build the model, while inference runs continuously to serve real users. - Batch vs real-time inference Batch is cost-efficient for large workloads, while real-time is critical when user experience depends on instant responses. - Accuracy vs reliability Accuracy looks good on test data, while reliability shows consistent performance under real-world conditions. - Guardrails vs validation Guardrails prevent unsafe outputs, while validation ensures correctness. Both are needed for safe and dependable systems. - Offline vs online evaluation Offline testing uses past data, while online evaluation measures real user impact. One doesn’t guarantee the other. - Data drift vs model drift Data drift changes inputs, while model drift shows performance degradation. Detecting this early avoids silent failures. - Monitoring vs observability Monitoring tracks known issues, while observability helps you understand unknown failures and system behavior. - Model hosting vs model serving Hosting deploys the model, while serving handles scaling, routing, and real-time requests. This is where complexity grows. - RAG vs fine-tuning RAG brings in fresh external knowledge, while fine-tuning embeds knowledge into the model. One adapts, the other is fixed. - Latency vs throughput Latency is response speed, while throughput is volume. Systems often fail because latency becomes too high. - Prompting vs fine-tuning Prompting shapes behavior through instructions, while fine-tuning changes model weights. Many real systems rely more on prompting. Understanding these trade-offs is what makes AI systems actually work. Which of these has been the toughest in your production setup?