Nick Bavafa

Why "On-Device" is the most ethical AI strategy. We talk a lot about the size of models. Parameters. Trillions of tokens. The race for AGI. But in my academic life at EPFL and my work in Speech, I’ve always been obsessed with the opposite end of the spectrum: Accessibility. When we published our research in Nature Communications on AI for social projects, one conclusion was clear: If AI requires a massive data center and a high-speed fiber connection to run, it excludes half the world. Cloud models are undeniably impressive and they are pushing the boundaries of what is possible. But they shouldn't be the only way we deploy intelligence. I'm seeing a resurgence of interest in on-device solutions, not to replace the cloud, but to handle the sensitive, immediate, and personal tasks where the edge shines: - Privacy: Processing data right where it’s created. This is crucial for multimodal data (voice, video) that is highly identifiable and shouldn't leave the user's hand. - Latency: Real-time processing that feels like a conversation, not a transaction. - Energy Efficiency: Running a small, optimized model locally is often far more sustainable than round-tripping every query to a power-hungry data center. - Reliability: True accessibility means working offline. AI should be helpful even when the signal drops or the fiber hasn't been laid yet. Better locally-run alternatives make these powerful tools accessible to everyone, even for problems and data that are highly sensitive. I’m planning to dive deeper into this myself by experimenting with some of the latest open-weights models specifically for local deployment. If you’ve been testing small language models (SLMs) or edge-optimized architectures recently, I’d love to hear about your experience. What frameworks or models are you finding most promising for mobile/edge/small server hardware right now? #AIforGood #OnDeviceML #NLP #EthicalAI #TechForGood #EdgeAI #SustainableTech #OpenWeights

7

6 Comments

🚀 If you’re still feeding your LLMs with plain JSON, you might be wasting tokens, speed, and budget. Last week, TOON (Token-Oriented Object Notation) dropped — and it’s a real game changer for anyone building with LLMs. Here’s why TOON stands out 👇 ✅ Token-efficient: Reduces token usage by 30–60% compared to JSON for uniform arrays. ✅ LLM-optimized: Uses explicit lengths and field sets to improve parsing and retrieval accuracy. ✅ Minimal syntax: No redundant braces, quotes, or repeated keys — meaning faster comprehension. ✅ JSON-compatible: Build in JSON, convert to TOON for LLM use, and switch back anytime. 💡 When to use TOON Perfect for large, structured datasets — like telemetry logs, user data, or product catalogs. For deeply nested or irregular structures, JSON might still be the better fit. If you’re developing agents, copilots, or multi-tool LLM systems, it’s time to make TOON your go-to serialization format. Your token budget, throughput, and accuracy will thank you! ⚡ Would you like me to make it more technical (for LLM engineers) or LinkedIn-engaging (for broader audience appeal)?

14

1 Comment

The $100k question: Where will AI talent go next? The US just slapped a $100k toll on new H-1B visas. A signal as loud as it is simple: foreign talent, you’re welcome here… if you can afford the luxury lane. The headlines focus on costs for companies. But the deeper story is about loyalty, geography, and the future of AI "clusters". So let’s pull on a few threads: [1] Is talent loyalty shifting from “I’ll go where the money is” to “I’ll go where the system values me”? For decades the US was the unquestioned magnet. The best labs, the biggest paychecks, the prestige of “making it in America” But loyalty is fragile. If the U.S. turns visas into barriers, talent won’t just chase salaries - it will chase respect, speed, and stability. A gifted AI PhD might ask: - Do I want to gamble $100k on bureaucracy? - Or go to a country that treats me as an asset, not a liability? Money matters. But dignity travels further than dollars. [2] Will the US become less a magnet and more a barrier - and will that accelerate the decentralisation of AI innovation? Every moat eventually becomes a wall. The US is still rich in capital and infrastructure. But barriers shift incentives: - Startups may seed themselves in London, Toronto, Berlin, or Singapore - Researchers may take UK fast-track visas over H-1Bs (recently announced) - Remote-first labs may bypass US soil entirely The more America closes the door, the more others will rush to build the house. [3] Could jurisdictions become hyper-specialised? Imagine a new world map of AI: - Europe: research powerhouses, anchored in universities and public funding - Asia: rapid deployment, scale, and applied systems - UK: financial hub for AI investment and governance - US: still dominant, but increasingly selective, more "Silicon Valley for the elite" If so we’re not heading for one global AI capital - we’re heading for an cluster of specialisations. Visa policies aren’t just about headcount. They shape where knowledge flows, where ecosystems take root, and where people feel they belong. The US may still have the deepest pools of capital and the strongest track record of innovation, but talent is increasingly mobile. Systems that offer clarity, dignity, and long-term opportunity could matter just as much as paycheques. Perhaps that’s no bad thing. A decentralised AI world could reduce concentration risk, spread innovation across more societies, and allow countries to play to their relative strengths - research, deployment, governance, investment. The $100k fee is more than a price tag. It’s a reminder that where AI is built will depend less on who pays the most, and more on who creates the best environment for talent to thrive.

47

22 Comments

LLMs will acceletrate a lot of good tech and data trends that I think haven’t gotten as much traction as they deserve: * OSS, I think we’ll see open core get new life (especially as a way to get “into” the LLM training data) * code-first interfaces (e.g. code first BI like I mentioned in a recent post) * _really_ good CLIs and APIs This is all especially important for leveling up developer workflows and experience — if your tools aren’t open, don’t have code interfaces, and don’t have a CLI, it’s very hard to imagine how you succeed in an LLM world

2

Seed vs. Series C ML hiring: Same title, completely different humans Seed ($10M raised): Profile: Former founding engineer, 0-to-1 at scale-up, wore every hat Interview tests: Speed, ambiguity tolerance, "build the plane while flying it" Comp: $140-180K base, 0.5-2% equity, you're betting on yourself Series C ($100M+): Profile: FAANG or late-stage startup specialist, deep in one domain Interview tests: Depth, cross-functional polish, "improve the machine" Comp: $250-320K base, 0.1-0.3% equity, you're optimizing a working system The friction I see weekly: Seed founders rejecting FAANG vets for "needing too much structure" Series C teams rejecting startup generalists for "not deep enough" Neither is wrong. Both are mismatched. First question I ask founders: "Which game are you actually playing?" #hiring #startups #seed #funding

52

2 Comments