Impact of Generative AI

NEWS 21/10/25: Department of Homeland Security obtains first-known warrant targeting OpenAI for user prompts in ChatGPT According to a recent article by Forbes, the U.S. Department of Homeland Security (DHS) has secured a federal search warrant ordering OpenAI to identify a user of ChatGPT and to produce the user’s prompts, as part of a child-exploitation investigation. https://lnkd.in/eatmK3zv? Key details: - The warrant was filed by child-exploitation investigators within DHS. - It specifically targets “two prompts” submitted to ChatGPT by an anonymous user. The warrant asks OpenAI for the user’s identifying information and associated prompt history. - This is described as the first known federal search warrant compelling ChatGPT prompt-level data from OpenAI. What this means for privacy: -Prompts are treated as evidence. What users have assumed to be ephemeral or private entries in a chat session with an AI service may now be subject to law-enforcement production. -Scope of data retention and access must be reconsidered. If prompt history can be identified and requested, both users and providers should evaluate how long prompts are stored, under what identifiers, and how anonymised they truly are. - Implications for user trust and provider responsibility. AI companies may face growing legal obligations to disclose user-generated content and metadata, which may affect how the services present themselves (privacy guarantees, terms of service) and how users engage with them. - International context and legal cross-overs. For users in jurisdictions with strong data-protection regimes (for example, the General Data Protection Regulation in the UK/EU), the fact that prompt-data can be subject to U.S. warrant may raise questions about extraterritorial access and data flow compliance. In short: this isn’t just another law-enforcement request. It marks the first time a generative-AI provider has been legally compelled to unmask a user and disclose their prompt history. ============ ↳I track how stories like this shape the ethics and governance of AI. You can find deeper analysis at discarded.ai. #AISafety #AIRegulation #Privacy #Governance #Ethics Image AI Generated

ChatGPT’s new reasoning models are hallucinating more often than previous generations, and the cause is still under investigation. It appears that adding the ability to break tasks down degraded OpenAI’s LLM reliability. On the PersonaQA and SimpleQA benchmarks, OpenAI’s new reasoning models (o3 and o4-mini) hallucinated between 33% and 79% of the time. The problem is likely to be impacting Google and DeepSeek’s reasoning models. It may be a cascading failure where multiple calls to the LLM amplify minor issues and inaccuracies as more steps are completed. The result of piling all those small inaccuracies on top of each other could be more noticeable errors. In any case, reasoning models fail at rates that make them unusable for consumer-facing products. It’s another setback to productizing LLMs, and there’s no timeline for when reliability will improve. For now, small language models (SLMs) are the best option for generative AI products. They cost less and are easier to put guardrails around. Post-training SLMs with domain-specific data helps them achieve higher reliability than LLMs. SLMs lack the horizontal breadth of knowledge but make up for it with vertical depth, enabling a narrow set of capabilities. They can support a few workflows well, but don’t generalize like LLMs are intended to. However, LLMs don’t meet the reliability requirements for most use cases. When they generalize, users can’t trust the output, so they can’t be integrated into AI products, especially agents that take action independently. As Anthropic recently discovered, we can’t trust the LLM’s explanations of how they arrived at the answers and output they generate. LLMs will often provide an explanation that doesn’t fit the reality of their internal processes. The fact that LLMs are unexplainable and unreliable means they aren’t ready for prime time. That doesn’t mean the technology is useless, and it’s essential not to overlook what does work (SLMs) just because some things don’t.

We just released a new 90-page market study covering the Generative AI Market globally. 🧠 Generative AI (GenAI) is a novel technology that enables computer systems to produce original, human-like content across text, images, video, audio and software code. The market is evolving rapidly as more capable and intelligent models are continuously being announced. The market is not yet a winner-takes-all; low switching costs have resulted in a commoditised and fragmented landscape. However, companies can still differentiate through unique training methodologies that produce distinct output styles and “personalities”. Berg Insight expects the influx of new GenAI companies to continue in the coming years, followed by a phase of consolidation within three to five years. The winning companies will be those that can leverage strong financial backing while attracting top-tier talent to drive rapid innovation. 📈 In 2024, the GenAI market experienced triple-digit growth rates in all three major segments spanning GenAI hardware, foundation models and development platforms. The market value for foundation models reached an estimated US$ 4.1 billion, excluding end-user applications such as ChatGPT. The figure primarily includes income through API services or license fees as the models are used via development platforms. Meanwhile, the market value for GenAI development platforms reached an estimated US$ 17.0 billion. Furthermore, GPU-based hardware systems used for GenAI workloads generated revenues of US$ 132.3 billion in 2024. 🏢 Berg Insight has identified 31 key foundation model providers spanning LLMs, vision, audio and multimodal models. While many LLMs started as unimodal, nearly all successful LLMs now include multimodal capabilities. Companies with notable cross-modal offerings include US-based Anthropic, Google, Meta, OpenAI and xAI; China-based Alibaba Cloud, Baidu, Inc., ByteDance and Tencent; France-based Mistral AI and Canada-based Cohere. Specialised vision model developers include US-based Midjourney and Runway, and UK-based Stability AI. In audio, key specialists include US-based AssemblyAI and ElevenLabs. The ecosystem is supported by a host of development platform providers offering tools for building GenAI applications. In the US, key providers include cloud giants like Microsoft, Google and Amazon Web Services (AWS), and diversified tech companies such as IBM and Oracle. The landscape also features hardware providers like NVIDIA, data platform specialists such as Databricks and Snowflake, model training platforms like Scale AI, and the open-source library from Hugging Face. European and Asian players also contribute, including Dutch Nebius and the aforementioned Chinese conglomerates. #AI #GenerativeAI #technology #innovation #marketresearch

Generative AI has taken “AI” out of the hands of specialists and placed it into every industry that has a problem to solve. When teams can build with natural language, integrate with existing systems, and map human intent instead of rigid filters, AI stops being a lab project and becomes a business capability. I was going through some recent case studies from Publicis Sapient and one of them really stood out to me. It captures something important about where we are in this GenAI wave. We finally have AI that is not limited to technical teams. It is being used directly to reshape customer experience in ways that people can actually feel. The Homes and Villas by Marriott Bonvoy project is a great example of this shift. Publicis Sapient and Marriott built a generative search experience using Azure OpenAI that turns natural language intent into real, bookable vacation homes. Not filters, not rigid queries. Actual human intent. A few technical details from the case study that I loved: ✦ Intent parsing over keyword search Travelers can describe feelings or preferences. The system uses LLMs to infer constraints, property attributes, and destination suggestions across 150K listings. ✦ GPT based retrieval pipeline LLMs enrich the query, expand candidates, and rerank results based on nuanced signals which reduces dead ends and increases high confidence matches. ✦ Real time context generation Weather, activities, and travel ideas are synthesized for each result which turns simple search into discovery. ✦ Enterprise scale rollout acceleration Once the pattern was built, Marriott cut expansion time from a year to three months which shows how GenAI lowers the cost of experimentation inside large organizations. If you want to dive deeper into the Marriott project and the system behind it, the full Publicis Sapient customer story is a great read: https://lnkd.in/evGBTTTN

Generative AI continues to generate excitement, but significant challenges are often overlooked. Reports from respected sources such as Harvard Business Review and Goldman Sachs highlight that current expectations may not align with reality. The technology, while promising, has limitations that need to be acknowledged and addressed. In May, Harvard Business Review discussed "AI's Trust Problem," in June, Goldman Sachs raised doubts about whether the expected $1 trillion in AI investment will deliver substantial returns. Their concern: aside from developer efficiency, there may not be enough value to justify such massive spending, especially in the near term. Jim Covello, Goldman Sachs' head of global equity research, pointed out that replacing low-wage jobs with costly technology contradicts earlier tech transitions, which focused on improving efficiency and affordability. A recent analysis from Planet Money echoes this skepticism, listing “10 reasons why AI may be overrated.” Issues like hallucinations (when AI generates false or misleading information) and declining quality in AI-generated outputs raise concerns about its readiness for widespread use. A study by The Washington Post also examined what people ask AI chatbots about, revealing unexpected trends. Along with common academic assistance, some topics raised ethical and personal concerns. 🔍 Reality check: Generative AI can be impressive but often struggles with accuracy, leading to errors or hallucinations. 💸 Investment risks: Financial experts question the value of massive investments in AI and wonder if the technology will offer enough returns in the short term. 📉 Productivity vs. quality: While AI can increase productivity, particularly in coding, research shows that the quality of AI-generated code is often subpar. 📚 Help with homework: Students turn to AI chatbots for homework help, but concerns arise when AI provides direct answers rather than guidance or learning support. ❓ Personal and sensitive queries: Many chatbot users ask about personal topics, including sex and relationships, which raises ethical questions about privacy and appropriate use. These points serve as a reminder that while generative AI is a powerful tool, it’s important to approach it with realistic expectations and a clear understanding of its current limitations. #GenerativeAI #AIEthics #AIRealityCheck #AIinEducation #TechInvestments #AIProductivity #AIChallenges #AIHomework #AIandSex #AIinConservation #AIFuture #AIHype 

Check out this massive global research study into the use of generative AI involving over 48,000 people in 47 countries - excellent work by KPMG and the University of Melbourne! Key findings: 𝗖𝘂𝗿𝗿𝗲𝗻𝘁 𝗚𝗲𝗻 𝗔𝗜 𝗔𝗱𝗼𝗽𝘁𝗶𝗼𝗻 - 58% of employees intentionally use AI regularly at work (31% weekly/daily) - General-purpose generative AI tools are most common (73% of AI users) - 70% use free public AI tools vs. 42% using employer-provided options - Only 41% of organizations have any policy on generative AI use 𝗧𝗵𝗲 𝗛𝗶𝗱𝗱𝗲𝗻 𝗥𝗶𝘀𝗸 𝗟𝗮𝗻𝗱���𝗰𝗮𝗽𝗲 - 50% of employees admit uploading sensitive company data to public AI - 57% avoid revealing when they use AI or present AI content as their own - 66% rely on AI outputs without critical evaluation - 56% report making mistakes due to AI use 𝗕𝗲𝗻𝗲𝗳𝗶𝘁𝘀 𝘃𝘀. 𝗖𝗼𝗻𝗰𝗲𝗿𝗻𝘀 - Most report performance benefits: efficiency, quality, innovation - But AI creates mixed impacts on workload, stress, and human collaboration - Half use AI instead of collaborating with colleagues - 40% sometimes feel they cannot complete work without AI help 𝗧𝗵𝗲 𝗚𝗼𝘃𝗲𝗿𝗻𝗮𝗻𝗰𝗲 𝗚𝗮𝗽 - Only half of organizations offer AI training or responsible use policies - 55% feel adequate safeguards exist for responsible AI use - AI literacy is the strongest predictor of both use and critical engagement 𝗚𝗹𝗼𝗯𝗮𝗹 𝗜𝗻𝘀𝗶𝗴𝗵𝘁𝘀 - Countries like India, China, and Nigeria lead global AI adoption - Emerging economies report higher rates of AI literacy (64% vs. 46%) 𝗖𝗿𝗶𝘁𝗶𝗰𝗮𝗹 𝗤𝘂𝗲𝘀𝘁𝗶𝗼𝗻𝘀 𝗳𝗼𝗿 𝗟𝗲𝗮𝗱𝗲𝗿𝘀 - Do you have clear policies on appropriate generative AI use? - How are you supporting transparent disclosure of AI use? - What safeguards exist to prevent sensitive data leakage to public AI tools? - Are you providing adequate training on responsible AI use? - How do you balance AI efficiency with maintaining human collaboration? 𝗔𝗰𝘁𝗶𝗼𝗻 𝗜𝘁𝗲𝗺𝘀 𝗳𝗼𝗿 𝗢𝗿𝗴𝗮𝗻𝗶𝘇𝗮𝘁𝗶𝗼𝗻𝘀 - Develop clear generative AI policies and governance frameworks - Invest in AI literacy training focusing on responsible use - Create psychological safety for transparent AI use disclosure - Implement monitoring systems for sensitive data protection - Proactively design workflows that preserve human connection and collaboration 𝗔𝗰𝘁𝗶𝗼𝗻 𝗜𝘁𝗲𝗺𝘀 𝗳𝗼𝗿 𝗜𝗻𝗱𝗶𝘃𝗶𝗱𝘂𝗮𝗹𝘀 - Critically evaluate all AI outputs before using them - Be transparent about your AI tool usage - Learn your organization's AI policies and follow them (if they exist!) - Balance AI efficiency with maintaining your unique human skills You can find the full report here: https://lnkd.in/emvjQnxa All of this is a heavy focus for me within Advisory (AI literacy/fluency, AI policies, responsible & effective use, etc.). Let me know if you'd like to connect and discuss. 🙏 #GenerativeAI #WorkplaceTrends #AIGovernance #DigitalTransformation

It’s easy to think of AI as a time-saver that streamlines workflows and accelerates output. But the deeper opportunity lies in how it’s reshaping the nature of work itself. A new study from Harvard Business School’s Manuel Hoffmann followed more than 50,000 developers over two years, with half using GitHub Copilot. The results were striking: developers shifted away from project management and toward the core work of coding. Not because someone told them to, but because AI made it possible. With less need for coordination, people worked more autonomously. And with time saved, they reinvested in exploration—learning, experimenting, trying new things. What we’re seeing here isn’t just productivity. It’s a shift in how work gets done and who does what. Managers may spend less time supervising and more time contributing directly. Teams become flatter. Hierarchies adapt. This is just one signal of how generative AI is changing our org charts and challenging us to rethink how we structure, support, and lead our teams. The future of work isn’t just faster. It’s more fluid. And if we get this right, it’s a whole lot more human. https://lnkd.in/gaUgXnRY

Recently, a Gartner study estimated that 30% of Generative AI projects will be abandoned after proof of concept by the end of 2025. In the study, Gartner highlights 4 key possible causes for the 30% of projects that will be dropped out to be either: 1- Poor data quality 2- Inadequate risk controls 3- Escalating costs 4- Ambiguous business value As #generativeAI starts to rate lower in the emerging tech “Hype Cycle” – an expected and common phenomenon in the tech world, users and vendors are collectively reaching a realistic stage of the technology adoption lifecycle, where the excitement about potential takes a backseat, and realities of cost, difficulties of implementation and the pressure of achieving business value become more prominent. In my view, this percentage is a realistic estimate given where Generative AI is at the moment. Continuous experimentation and the search for innovation and exciting use cases should continue, but on the same footing, organizations need to realistically assess Gen AI proposed projects to improve the chances that these projects will make it to wide scale adoption as opposed to being dropped out post POC. Focusing on the cost challenge above, one of the reasons I see many Gen AI projects fail is the high cost of deploying Large Language Models (LLMs). As exciting as #LLMs are, the costs associated with running them is high including computational resources needed to run them in terms of storage, data processing capabilities and other operational costs. In many cases, Small Language Models (#SLMs) can be a more effective choice for Gen AI projects due to their efficient use of computing resources, the ability to scale them faster and cheaper as well as the lower cost of training these models as they focus on smaller sets of parameters and data sources.

This Stanford study examined how six major AI companies (Anthropic, OpenAI, Google, Meta, Microsoft, and Amazon) handle user data from chatbot conversations.  Here are the main privacy concerns. 👀 All six companies use chat data for training by default, though some allow opt-out 👀 Data retention is often indefinite, with personal information stored long-term 👀 Cross-platform data merging occurs at multi-product companies (Google, Meta, Microsoft, Amazon) 👀 Children's data is handled inconsistently, with most companies not adequately protecting minors 👀 Limited transparency in privacy policies, which are complex and hard to understand and often lack crucial details about actual practices Practical Takeaways for Acceptable Use Policy and Training for nonprofits in using generative AI: ✅ Assume anything you share will be used for training - sensitive information, uploaded files, health details, biometric data, etc. ✅ Opt out when possible - proactively disable data collection for training (Meta is the one where you cannot) ✅ Information cascades through ecosystems - your inputs can lead to inferences that affect ads, recommendations, and potentially insurance or other third parties ✅ Special concern for children's data - age verification and consent protections are inconsistent Some questions to consider in acceptable use policies and to incorporate in any training. ❓ What types of sensitive information might your nonprofit staff  share with generative AI?  ❓ Does your nonprofit currently specifically identify what is considered “sensitive information” (beyond PID) and should not be shared with GenerativeAI ? Is this incorporated into training? ❓ Are you working with children, people with health conditions, or others whose data could be particularly harmful if leaked or misused? ❓ What would be the consequences if sensitive information or strategic organizational data ended up being used to train AI models? How might this affect trust, compliance, or your mission? How is this communicated in training and policy? Across the board, the Stanford research points that developers’ privacy policies lack essential information about their practices. They recommend policymakers and developers address data privacy challenges posed by LLM-powered chatbots through comprehensive federal privacy regulation, affirmative opt-in for model training, and filtering personal information from chat inputs by default. “We need to promote innovation in privacy-preserving AI, so that user privacy isn’t an afterthought." How are you advocating for privacy-preserving AI? How are you educating your staff to navigate this challenge? https://lnkd.in/g3RmbEwD

Most enterprise generative AI projects still struggle to show measurable financial returns within their first six months. That tolerance is fading because boards and investors now want AI to add to earnings instead of just serving as a test. The focus has shifted from pilots to impact on profits and losses. Spending on AI is increasing, while control over capital is getting stricter. Leaders who cannot link AI to better margins or increased revenue risk losing their budgets and credibility. What’s changing is how deployment is viewed. Early efforts were exploratory because the technology was new. Now, management teams are focusing on use cases that directly relate to reducing costs or improving measurable efficiency, as vague claims of productivity gains are no longer accepted. This means AI initiatives must connect to financial statements, not just innovation presentations. Another change is the emphasis on readiness. Only a small number of organizations consider their infrastructure or data environment to be ready for AI because outdated systems create obstacles. Companies that are using AI to upgrade their IT are saving money that they can use for further deployment, as improved efficiency builds on itself. This means modernisation and return on investment must progress together to maintain funding. Random or broad AI projects fail because they overlook workflow realities and data limitations. Targeted deployment focused on clear outcomes leads to measurable results. Measuring sentiment or perceived productivity does not work because boards care about contributions to earnings. Tracking costs and cycle times in workflows provides a solid basis for ROI. One good starting point is to choose a workflow that involves a practical starting point is a workflow with frequent decisions. Measure its cycle time and transaction costs first. Then introduce AI support. Avoid using AI in areas where data is scattered or governance is unclear because scaling up will be difficult. #AIROI #EnterpriseAI #AILeadership #DigitalTransformation #DataStrategy #CIO #CEOAgenda #BusinessValue #AIAdoption #TechStrategy #BoardGovernance #AITalent