Building Trust in AI Applications

Financial crime compliance may be enterprise AI's toughest proving ground. A model can be accurate and still be unusable if a compliance team cannot explain, govern, or audit its output. One interesting chat I had recently on this topic was with the Flagright team. They are building an AI operating system for financial crime compliance that brings  transaction monitoring, watchlist screening, risk scoring, case management, investigations, regulatory filing, and governance into one system. What I found interesting about their approach: ◾AI is embedded in real investigation workflows, not bolted on as a separate chatbot. ◾Teams can surface relevant evidence, investigate alerts, draft narratives, and recommend next steps while keeping human oversight. ◾Decisions remain explainable and audit-ready. ◾One operating layer reduces fragmented tools and handoffs. I think some of this thinking applies well beyond financial services too. AI in production does not just need to work. It also needs to be governed from the start and accountable when decisions matter. And congrats to the Flagright team on their $12.5M Series A, led by Infinity Ventures. 📍Worth exploring: https://lnkd.in/gGJAU7u6 #AI #FinancialCrime #Compliance

Satya Nadella described a future at Microsoft where there may be more than 20 million agents working alongside employees. This brings up interesting questions about how to monitor what these agents are doing, what these agents need to look like, and what they're allowed to access. Satya believes we need to start with the non-negotiables. Agents need to be fully inspectable and fully auditable. And the moment an agent can write code and execute it, that code has to run in an environment governed by policy. This is one of the engineering challenges of the AI moment; the infra that has to get built as companies stand up the platform for agentic work. If AI is going to amplify human capability at scale, we have to know what our agents are doing, constrain what they can access, and be able to audit and intervene when something goes wrong. It's what makes large-scale deployment possible. It's what earns trust. The alternative is launching millions of autonomous systems into production and hoping for the best.

🚨 Zero Trust for AI Agents Anthropic just released "Zero Trust for AI Agents." As we're thinking about agentic permissions, applying a Zero Trust discipline is critical to secure adoption. AI agents interpret goals, call tools, chain actions, delegate to other agents, and maintain context across sessions. The trust surface is different. The paper introduces "least agency" — a concept that OWASP has been promoting — and the distinction from least privilege is worth sitting with. 👉 "Least privilege" asks what an identity can access. 👉 "Least agency" asks what an agent can do, under what conditions, with which tools, and with what level of oversight. Autonomous agents introduce action risk alongside access risk — and the boundaries around behavior need to be architecturally enforced, not assumed. The paper includes a design test worth writing down: 🔥 Does the control make the attack impossible, or merely tedious?🔥 The practical controls follow directly from Zero Trust fundamentals — cryptographic agent identity, short-lived credentials, tool allow-listing, sandboxed execution, and full traceability from prompt to action to outcome. None of this is new doctrine. It's existing architecture applied to a harder problem. Full disclosure: the paper cites NIST SP 800-207 on Zero Trust Architecture and the CISA Zero Trust Maturity Model, both of which I co-authored during my time supporting Federal Zero Trust efforts at CISA. Zero Trust is built for a world where we have to remove implicit trust. Agentic AI is the next version of that same problem — valid identities, valid credentials, legitimate-looking actions, and still no basis for assumed trust. Access is earned. Actions are constrained. Agency must be governed. 👉🏼 Link to Anthropic's paper in the comments.

🪂 How To Make Your Design System AI-Ready (https://lnkd.in/dtnpy7CM), a practical guide on how to reduce drifts, minimize mistakes, maintain context and improve the quality of AI-generated prototypes — with structured spec files, automated auditing and token layers. Put together by Hardik Pandya from Atlassian. --- 🔹 1. Design Decisions Are Infrastructure AI-generated prototypes often don't deliver consistently decent results because of tiny inconsistencies scattered all across a design system. Often it's decisions made but not documented, hard-coded values never cleaned up, or relying too much on AI making sense of mock-ups or design flows on its own. Unsurprisingly, better AI prototypes come from better data — but also from better human guidance. We shouldn’t assume that AI knows how to choose the right component, and how to design with accessibility in mind. It needs priorities, a clear path on how we make decisions, design principles, examples, do's and don'ts. In fact, we should treat design decisions as infrastructure. That means that every time we make a decision — not just a design decision, but even decision on how actually prioritize our work and how we make decisions around here — it must find a path into the spec file that is then consumed by AI. --- 🔶 2. Three Layers: Spec Files + Token Layer + Audit To ensure quality, we establish design principles, guidelines, rules in a form of “spec files”). It's structured Markdown files that include spacing rules, color choices, component usage guidelines, priorities etc. AI is going to read and reuse that spec file every time it's going to generate a prototype. Because the spec files are text files, it's much more cost-effective, but also much more accurate just because we don't rely on AI recognizing or decoding patterns from mock-ups, but gets specific guidelines instead. In fact, extending code is often a more effective way than generating code from mock-ups. Token layer lists and keeps updated all tokens used throughout the design system. AI always chooses from a closed set of named variables instead of inventing plausible values ad-hoc. An audit script catches what AI gets wrong. It scans the prototype and flags every hard-coded value and flags it if necessary. It can be a regular software doing that, with AI waiting for its feedback to come back. Finally, when a design system ships updates, a sync routine flags which spec files need updating. The goal is to make sure that AI always reads up-to-date, current specs, not the ones written against an outdated version. --- 🔺 3. Examples of AI-Ready Design Systems ⌾ Atlassian: https://lnkd.in/dVsGc3Cp ⌾ Carbon: https://lnkd.in/d4zq4WWb ⌾ CMS Design System: https://lnkd.in/dHHzV3en ⌾ Nordhealth: https://lnkd.in/d8C4j2ZA Yet again, AI can’t magically resolve technical debt or design debt — it needs guidance, decisions, priorities and principles.

None will ever be able to stop Shadow AI! When people find an AI tool that helps them work faster, think better, or save time, they use it. That is what good people do. They look for leverage. Banning this is not a strategy. It is just an invitation to hide it, which is worse. So, what can we do? In my discussion with Stephen Schmidt, Chief Security Officer at Amazon, one message came through very clearly: 👉 The role of security is no longer to stop Shadow AI—because it is impossible. 👉 The role of security is to make AI safe, visible, and controlled. 👉 To know what is being used, where it is installed, what it can access, and where the data goes. That is the real shift. Because the biggest danger with AI is often not the intelligence. It is the permission. The moment an agent gets broad access to your files, systems, or sensitive data, your risk changes completely. So the question is not: “How do we stop Shadow AI?” The question is: “How do we make sure AI does not operate in the shadows?” That means four things: 1️⃣ Visibility: Create an inventory of the AI tools and agents people are actually using. 2️⃣ Boundaries: Run agents in isolated environments, not freely on laptops or production systems. 3️⃣ Permissions: Give agents only the minimum access they need, nothing more. 4️⃣ Traceability: Log actions so you know what the agent did, what data it touched, and who triggered it. This is where many leaders get it wrong. They think control means restriction. It does not. Real control means creating an environment where AI can be used fast, safely, and in the open. The companies that try to ban AI will lose visibility. The companies that learn to govern it will gain trust, speed, and advantage. You cannot stop Shadow AI. But you can stop unmanaged AI. 💥 Curious to learn more: https://lnkd.in/eubr-VpH How is your organization dealing with this today? #AWSAmbassador #AI #AgenticAI #Cybersecurity #Leadership #FutureOfWork

Over the course of my career, I’ve learned to be okay with getting things wrong.       Not because it feels good (it doesn’t), but because every mistake creates an opportunity to learn and grow. And because it means someone trusted me enough to tell me when I missed the mark. That kind of honesty feels increasingly rare—especially in a world where AI is telling people exactly what they want to hear and where people increasingly gravitate toward information that confirms their beliefs.      That’s why I think one of the most valuable skills you can cultivate is this: Find people who will give you tough feedback.      Across my time at Microsoft, the Gates Foundation, and Pivotal, the moments that shaped me the most weren’t the wins. They were the times when someone I trusted pulled me aside and gave me feedback I needed to hear. These conversations helped me see what I’d missed and rethink how I was showing up, which made me a better leader. But they only happened because the people around me knew they could be honest, and in fact, I expected them to be. You can’t grow—or help your teams grow—if you act like you’re the only one with all the answers.      I’ve seen this in every place I’ve worked. The leaders who made the biggest impact weren’t the ones who got it right all the time. They were the ones who created the conditions for honesty. Their teams felt free to surface new ideas, ask tough questions, and admit their mistakes. And those leaders were humble enough to hear feedback about themselves—and then take the steps to do things differently.       My advice on how to build this skill? Seek out colleagues and mentors you can trust to give you honest feedback. Ask for it often. Be vulnerable—not defensive—and take the opportunity to understand what you didn’t see before. It will transform the way you learn, lead, and build teams that thrive. #SkillsontheRise 

AI security/securing the use of AI is going to kill me. I use Claude Code almost daily. It's a problem.... Here's what I have to change AGAIN this week. Security researcher Ari Marzuk disclosed 30+ vulnerabilities across AI coding tools. Cursor. GitHub Copilot. Windsurf. Claude Code. All of them. He called it IDEsaster. The attack chain includes prompt injection, hijacking LLM context, and auto-approved tool calls executing without permission. Then, legitimate IDE features are weaponized for data exfiltration and RCE. Your .env files. Your API keys. Your source code. Accessible through features you thought were safe. Most studies I read claim that around 85% of developers now use AI coding tools daily. Most have no idea their IDE treats its own features as inherently trusted. 𝗦𝗼... 𝗮𝗳𝘁𝗲𝗿 𝗿𝗲𝘃𝗶𝗲𝘄𝗶𝗻𝗴 𝗔𝗿𝗶'𝘀 𝗿𝗲𝘀𝗲𝗮𝗿𝗰𝗵, 𝗵𝗲𝗿𝗲'𝘀 𝗜 𝘄𝗶𝗹𝗹 𝗯𝗲 𝗱𝗼𝗶𝗻𝗴... Be warned: All this is SO much easier said than done! Audit every MCP server connection. Checked for tool poisoning vectors where legitimate tools might parse attacker-controlled input from GitHub PRs or web content. Removed servers I couldn't verify. Disabled auto-approve for file writes. The attack chains weaponize configuration files and project instructions like .claude/settings.json and CLAUDE.md. One malicious write to these files can alter agent behavior or achieve code execution without additional user interaction. Move all credentials to a secrets manager. No .gitignored .env files in agent-accessible directories. API keys live in 1Password CLI. Environment variables inject at runtime through a wrapper script the LLM never sees. Start running Claude Code in isolated containers. Mounted volumes limited to specific project directories. No access to ~/.ssh, ~/.aws, or ~/.config. If the agent gets compromised, blast radius stays contained. Enable all security warnings. Claude Code added explicit warnings for JSON schema exfiltration and settings file modifications. These exist because Anthropic knows the attack surface. Add pre-commit hooks for hidden characters. Prompt injections hide in pasted URLs, READMEs, and file names using invisible Unicode. Flag non-ASCII characters in any file the agent might ingest. The fix isn't to stop using AI coding tools. The fix is to stop trusting them implicitly. What controls do you have for AI tools with write access to your codebase? 👉 Follow for more AI and cybersecurity insights with the occasional rant #AISecurity #DevSecOps

OpenAI just launched an Agent Builder. Say goodbye to thousands of startups AI is progressing so fast, most founders don't know how to build defensibility. Here’s what the smartest founders do differently: 1. They own the data APIs are temporary. What compounds is your proprietary dataset. → throxy (yc x25) scrapes its own data instead of relying on LinkedIn or Apollo. Every interaction improves the product and makes it harder to replicate. 2. They build a UX moat When tech is commoditized, experience becomes the differentiator. → Granola is a meeting recorder, but what sets it apart is the interface. It feels delightful, fast, frictionless—like it was made for you, not for enterprise IT. In a world of clones, great UX builds loyalty. 3. They unlock power for non-technical users Defensibility comes from enabling outsiders to do what only experts could before. → Lovable turns brand design into something anyone can do. In minutes, non-designers generate entire visual identities. 4. They move up the stack Commoditized infra dies. Winners package it into workflows users already need. → Perplexity started as a research tool. Now it’s becoming an AI-native browser. Owning the user’s daily search habits is how you defend long-term. 5. They go deep, not wide Horizontal tools get replaced fast. But narrow use cases can dominate markets. → ElevenLabs focused on voice. It didn’t try to do everything—just build the best voice AI in the world. Now it powers thousands of creators and companies. What am I missing? Full breakdown in my last article: https://lnkd.in/dSbpWnBz

Microsoft just released a 35-page report on medical AI - and it’s a reality check for healthcare. The paper, “The Illusion of Readiness”, tested six of the most popular models (OpenAI, Gemini, etc)… across six multimodal medical benchmarks. And the verdict? The models scored high on medical exams. But they’re not even close to being real-world ready. Here’s what the stress tests revealed: ▶ 1. Shortcut learning Models often answered correctly even when key information, like medical images, was removed. They weren’t reasoning - they were exploiting statistical shortcuts. That means benchmark wins may hide shallow understanding. ▶ 2. Fragile under small changes Making small tweaks caused big swings in predictions. This fragility shows how unreliable model reasoning becomes under stress. In visual substitution tests, accuracy dropped from 83% to 52% when images were swapped - exposing shallow visual–answer pairings. ▶ 3. Fabricated reasoning Models produced confident, step-by-step medical explanations - but many were medically unsound… or entirely fabricated. Convincing to the eye, dangerous in practice. And more importantly, healthcare isn’t a multiple-choice exam. It’s uncertainty, incomplete data, and high stakes. So Microsoft’s team calls for new standards: - Stress tests that expose fragility - Clinician-guided guidelines that profile benchmarks - Evaluation of robustness and trustworthiness - not just leaderboard scores The takeaway is simple: Medical AI may ace tests today. But until it proves reliable under stress, it’s not ready for the clinic. When do you think popular LLMs will be clinic-ready? #entrepreneurship #healthtech #AI

Uber’s internal on-call copilot, Genie, started with a RAG pipeline that sipped from 40+ security‑and‑privacy policy documents ( PDFs, Google Docs, wikis.... ) and served queries in Slack. Useful but far from reliable. SMEs flagged accuracy gaps. Answers were incomplete, context lost, tables misread. They didn’t reboot. They upgraded. ⭐ Enter Enhanced Agentic‑RAG (EAg‑RAG) 𝗘𝗻𝗿𝗶𝗰𝗵𝗲𝗱 𝗗𝗼𝗰𝘂𝗺𝗲𝗻𝘁 𝗣𝗿𝗼𝗰𝗲𝘀𝘀𝗶𝗻𝗴 They left behind flaky PDF parsers. They shifted to HTML‑formatted Google Docs, then layered in LLM‑powered enrichment: converting tables into clean markdown, injecting metadata as summaries, FAQs, keywords, identifiers. Each document chunk became searchable intelligence, not just contextless text. 𝗔𝗴𝗲𝗻𝘁𝗶𝗰 𝗥𝗔𝗚 𝗔𝗻𝘀𝘄𝗲𝗿 𝗚𝗲𝗻𝗲𝗿𝗮𝘁𝗶𝗼𝗻 They added agents before and after retrieval: • Query Optimizer breaks down or clarifies vague questions • Source Identifier narrows which docs matter • BM25 + vector search surfaces the best chunks • Post‑Processor de‑dupes and reorders context logically This combo meant go‑to‑answer accuracy shot up +27%, wrong advice dropped 60% all within critical security & privacy channels. 𝗟𝗟𝗠‑𝗮𝘀‑𝗝𝘂𝗱𝗴𝗲 They broke the SME bottleneck. By feeding queries, responses, and benchmarks to an LLM judge, they achieved automated batch evaluation which aligned with SME standards in minutes, not weeks. Why this matters: Operational AI is about tooling so reliable that SMEs use it without second‑guessing. Uber didn’t train a model but they layered structure, signals, and evaluation to treat knowledge as an operating system, not just data. They made the documentation better. They made the retrieval smarter. They made evaluation instantaneous. That’s systems thinking. That’s what separates an AI pilot from a one‑and‑done prototype. Check this gem in the Uber Engineering Blog post: https://lnkd.in/gUFDt9ug The image below shows end-to-end workflow of EAg-RAG architecture of this implementation