The Open-Source Compiler That Lets AI Run on Encrypted Data

Google Just Open-Sourced HEIR: A Compiler That Lets AI Read Your Data Without Ever Seeing It

Published August 16, 2026

On August 14, 2026, Google published a blog post that quietly describes one of the stranger promises in modern computing: a server can now process your data, run an AI model against it, and hand back a useful answer, all without ever knowing what your data actually says. The tool behind that claim is called HEIR, short for Homomorphic Encryption Intermediate Representation, and Google has released it as a free, open-source compiler on GitHub.

If that sounds like science fiction, you're not alone. The announcement collected hundreds of comments on Hacker News within a day, with cryptography-savvy readers split between calling it a genuine milestone for private AI inference and calling it years away from anything you'd actually want to ship. Both reactions are grounded in real numbers, and this article walks through exactly what Google built, how homomorphic encryption works in plain English, what the early demos actually show, and where the technology is still nowhere close to ready.


Table of Contents

What Google Actually Announced on August 14

The announcement came from Jeremy Kun, a staff software engineer at Google, in a post on the official Google blog titled "How Google is Making Private AI Practical with Homomorphic Encryption." The post frames HEIR as the newest addition to Google's Private Computing Toolkit, a family of privacy technologies that already includes differential privacy, private set membership, private information retrieval, and confidential computing on Google Cloud.

According to the post, HEIR can take a pre-trained AI model, one built and trained the normal way on unencrypted data, and convert it into a version that operates entirely on encrypted inputs. Google describes its long-term goal as making HEIR a one-click solution so that developers without a cryptography background can add encrypted inference to production applications.

The project itself is not brand new. Google first announced its intention to build HEIR back in 2023 as part of an expansion of its fully homomorphic encryption toolkit. What changed on August 14, 2026 is that the compiler has matured into something Google is now actively promoting with real partners, real hardware collaborations, and four working demo applications, each compiled end-to-end with HEIR and published with source code on GitHub.

Since 2023, HEIR has also become a research platform in its own right. Google says it has collaborated with teams at Georgia Tech, Carnegie Mellon, UC Santa Barbara, Illinois Institute of Technology, Purdue, the University of Edinburgh, and Tsinghua University, and that four peer-reviewed papers have already been built on top of the framework, with more in progress.

Why This Matters Right Now

AI adoption in 2026 has run headfirst into a privacy problem that isn't going away. Every cloud-based AI feature, from spam filters to fraud detection to content recommendations, requires a server somewhere to see your raw data before it can act on it. That's fine for low-stakes use cases. It's a serious liability for healthcare records, financial transactions, biometric data, and anything covered by regulations like HIPAA or Canada's PIPEDA.

The industry has tried a few workarounds. Running AI models locally on your device avoids sending data to a server, but it's limited by whatever hardware you're carrying and it risks exposing the company's proprietary model weights if the model itself ships to the device. End-to-end encryption protects data in transit and at rest, but as Google's own blog post points out, it breaks any feature that needs to actually read the data, like spam detection or virus scanning.

Homomorphic encryption is the third option, and it's the one nobody could make practical until recently. The timing lines up with a broader shift toward stricter data-handling rules across North America and the EU's ongoing AI Act enforcement, both of which are pushing companies to prove they can deliver AI features without touching sensitive data directly.

Homomorphic Encryption, Explained Without the Math

Here's the simplest way to think about it. Normal encryption is like putting a letter in a locked box. Nobody can read it until someone with the key opens it. Homomorphic encryption is like a locked box with built-in gloves attached to the outside. Someone can reach into those gloves, rearrange or process what's inside, seal it back up, and hand you a new locked box, all without ever opening it themselves.

In technical terms, homomorphic encryption lets a computer perform mathematical operations, like addition and multiplication, directly on ciphertext. When the result is later decrypted by whoever holds the private key, it matches exactly what you'd get from running the same operations on the original, unencrypted data.

Google's blog post gives a concrete example: a cloud service could generate personalized content recommendations without ever being able to see the user features it's basing those recommendations on. The server does real computational work. It just never gets to look at the inputs or outputs in plain form.

This isn't a new idea. Cryptographers have chased "fully homomorphic encryption" since the late 1970s, when Ron Rivest, Leonard Adleman, and Michael Dertouzos first floated the concept, though their early schemes turned out to be insecure. A workable version wasn't demonstrated until Craig Gentry's breakthrough dissertation in 2009. The gap between "mathematically possible" and "practical enough to ship" is exactly what HEIR is trying to close.

Inside HEIR: How the Compiler Actually Works

Built on MLIR, Not From Scratch

HEIR is built on top of MLIR, a compiler infrastructure originally developed for machine learning workloads and now widely used across the industry for building custom compilers. Rather than inventing an entirely new toolchain, Google's team layered homomorphic-encryption-specific abstractions on top of MLIR's existing framework. That decision matters because it gives HEIR compatibility with a well-supported ecosystem instead of forcing every contributor to learn a bespoke system.

A Python Entry Point for Non-Cryptographers

The most developer-friendly piece of HEIR is its Python frontend, distributed as the heir_py package. A developer writes a function in Python, marks which inputs are secret using type annotations, and HEIR handles compiling that function down through multiple layers of abstraction until it produces working homomorphic encryption code. The compiler defaults to the BGV encryption scheme and the OpenFHE backend, though it also supports the Lattigo backend for teams that prefer it.

Multiple Layers, Multiple Optimization Points

Academic documentation on the project describes HEIR as defining several layers of abstraction for homomorphic computations, each one designed to make a specific class of optimization easier to implement. That includes decisions like how ciphertext data gets packed together, how encryption parameters are selected, and how a program's logic gets translated, or "arithmetized," into a form the encryption scheme can actually execute. Because each layer is modular, cryptography researchers can focus on a single optimization technique and plug it into HEIR's existing testing and benchmarking infrastructure instead of rebuilding an entire compiler from zero.

Hardware Acceleration Partners

Software alone won't make homomorphic encryption fast enough for most real-world workloads, which is why Google has been working directly with companies building dedicated hardware accelerators for the technology, including Belfort Labs, Niobium, Cornami, and Optalysys. Google says it plans to publish latency benchmarks showing how much these accelerators improve on the CPU-only numbers from its initial demos.

The Four Demo Applications Google Shipped

Google didn't just release a compiler and a set of documentation. It shipped four working applications, each compiled through HEIR, with source code published publicly. All latency figures below come from single-threaded CPU testing, meaning real-world performance with dedicated hardware should eventually be faster.

1. Deep Learning Recommendation Model

Built jointly with Belfort Labs, LG, and New York University, this demo compiles a recommendation model so a service can generate content suggestions without ever seeing the user features driving the recommendation. This is the closest of the four demos to a mainstream consumer use case.

2. Credit Card Fraud Detection

Developed with Niobium and hardshell.ai, this demo compiles a fraud detector capable of flagging suspicious transactions without the processing server ever seeing the underlying transaction details in plain text, a direct answer to the kind of financial data exposure that regulators increasingly scrutinize.

3. Encrypted Network Threat Detection

Working again with Niobium, Google compiled the Kitsune anomaly-detection system to identify unusual patterns in encrypted network traffic. That means a security provider could monitor a client's network for intrusions without ever inspecting the actual contents of the packets flowing through it.

4. Hotword Detector

Built with Belfort Labs, this demo compiles a hotword detection model, the kind of system that listens for a wake word like "Hey Google" on a smart speaker. Running it homomorphically means an audio-triggered AI agent could recognize its trigger phrase while keeping the surrounding audio recording cryptographically protected.


Real-World Impact: Who Benefits First

The realistic near-term winners are organizations that already have a strong regulatory or reputational reason to avoid ever touching raw customer data, even internally. That points squarely at three sectors.

Healthcare: Diagnostic AI tools that never decrypt patient records could sidestep a huge share of HIPAA compliance overhead, since the server processing the data literally cannot expose what it never had access to in the first place.

Financial services: Fraud detection, credit scoring, and anti-money-laundering models are natural fits, since banks are often legally barred from freely sharing raw transaction data even with trusted partners. Multiple homomorphic encryption vendors are already piloting exactly this kind of cross-institution fraud model today.

Advertising and recommendation systems: As cookie deprecation and privacy regulation squeeze traditional ad targeting, encrypted inference offers a path to keep personalization working without centralizing raw user behavior data on an ad platform's servers.

For the average consumer, the more immediate benefit is indirect. You likely won't interact with a "homomorphic encryption" feature by name any time soon. Instead, you'll notice it as a checkbox in a privacy policy or a claim in a security whitepaper, evidence that a company's AI feature was built to never see your raw data at all.

The Catch: Cost, Speed, and the Hacker News Pushback

Google's own blog post doesn't hide the tradeoff. It describes homomorphic encryption as carrying a "nontrivial cost overhead" and frames the whole value proposition as shifting the privacy conversation from a hard capability limit to a question of cost, one that Google says is falling quickly.

That framing drew sharp pushback on Hacker News, where the announcement thread collected 384 points and 225 comments within a day or two of publication. Commenters with backgrounds in privacy-preserving machine learning pointed out that homomorphic encryption and related cryptographic techniques still carry overhead on the order of hundreds to a thousand times slower than equivalent operations on unencrypted data, depending on the operation type. Independent benchmarking discussions referenced elsewhere put simple encrypted integer operations at tens to hundreds of milliseconds each, with more complex operations like division running into multiple seconds, numbers that are workable for narrow batch tasks but far from anything approaching real-time interactive use.

To be fair, this is a well-known and long-standing limitation of the entire homomorphic encryption field, not a flaw unique to HEIR. Independent research summarizing recent FHE-based inference work found slowdowns of roughly 200 times on models like ResNet-20, largely because nonlinear operations, the kind neural networks depend on heavily, have to be approximated using polynomial functions that homomorphic encryption schemes can actually compute. Every workaround costs additional computation.

What the Developer Community Is Saying

Reaction across Hacker News, LinkedIn, and X in the days following the announcement fell into a few consistent camps.

Cautious optimism from cryptography researchers. Commenters with direct experience building privacy-preserving machine learning systems generally welcomed HEIR as a genuine step forward for interoperability, since it gives researchers a shared platform to compare optimization techniques instead of each lab building incompatible one-off compilers. Security commentator Amanda Walker, writing on LinkedIn about the release, described it as part of Google's ongoing effort to explore practical applications of fully homomorphic encryption, while noting that continued advances in computation speed and hardware acceleration are starting to make some use cases viable for broader adoption.

Skepticism about "practical" claims. A recurring theme in the Hacker News thread was frustration with Google's framing that cost is "rapidly decreasing." Commenters pushed back that the underlying overhead of homomorphic operations remains extremely high compared to plaintext computation, and that without dedicated accelerator hardware becoming widely available, most of these gains stay theoretical for typical engineering teams.

Genuine curiosity from application developers. A smaller but notable group of comments came from developers outside the cryptography field who were simply excited that a Python-first entry point now exists at all. For teams that previously assumed homomorphic encryption required a dedicated cryptography hire, a pip-installable package that handles the heavy lifting is a meaningfully lower barrier to experimentation, even if production deployment is still a long way off for most of them.

What people are hoping for next. Several commenters specifically asked when Google would publish real latency numbers using the hardware accelerator partnerships mentioned in the post, since CPU-only benchmarks are widely seen as a floor rather than a realistic production estimate. Others asked whether HEIR would eventually support larger transformer-based models rather than the smaller, well-scoped demos shown at launch.

HEIR vs. Other Private AI Tools

HEIR doesn't exist in a vacuum. Several other organizations have released homomorphic encryption libraries and compilers over the past few years, each with a different focus. Here's how the major options compare as of August 2026.

[Comparison Table]

Tool Maker Pricing Strengths Weaknesses Best For
HEIR Google Free, open source MLIR-based, multi-scheme and multi-hardware interoperability, Python frontend, active academic partnerships Still CPU-benchmarked publicly, larger models like LLMs not yet demonstrated Teams wanting a research-grade, extensible compiler platform
Microsoft SEAL Microsoft Research Free, MIT license Mature, widely deployed, tight Azure integration, strong documentation Lower-level library, requires more manual cryptographic expertise to use well Enterprise teams already inside the Azure ecosystem
Zama Concrete ML Zama Open source for research; commercial license required for production Scikit-learn and PyTorch-style APIs, strong ML developer ergonomics Commercial use requires a separate patent license ML teams who want the fastest path from a familiar Python model to FHE
OpenFHE Duality, Intel, and academic partners Free, open source Broad scheme support, strong academic pedigree, active development Lower-level; steeper learning curve for application developers Cryptography researchers benchmarking new schemes
Duality SecurePlus Duality Technologies Commercial, contact for pricing Enterprise support, multi-party encrypted collaboration, DARPA-backed research roots Not open source; higher cost of entry for smaller teams Regulated enterprises needing vendor support and SLAs

The short version: if you want a fully managed, enterprise-supported product, Duality or Microsoft SEAL are the safer bets today. If you want the most Python-friendly on-ramp for a machine learning team specifically, Zama's Concrete ML is hard to beat. If you're building research infrastructure or want a compiler designed to stay interoperable across encryption schemes and future hardware, HEIR is the one to watch.

Market Analysis: Homomorphic Encryption Is Becoming a Real Industry

This release doesn't happen in isolation. The global homomorphic encryption market has crossed roughly $250 million in 2026, according to industry research, with growth increasingly driven by post-quantum security requirements and regulatory pressure that make encrypted computation less of an experiment and more of a compliance necessity. Fully homomorphic encryption specifically is estimated to represent around 42 percent of that market this year, while cloud data security use cases account for roughly a third of total demand.

Key players tracked across multiple market reports include IBM, Microsoft, Google, Intel, Zama, Duality Technologies, Enveil, Cornami, and CryptoLab, alongside academic and government-adjacent research groups. Zama, notably, reached unicorn status in 2025 with a valuation above $1 billion after raising a Series B round, a signal that investors are betting real money on encrypted computation becoming commercially mainstream rather than staying a research curiosity.

Recent moves elsewhere in the space reinforce that momentum. In June 2026, Duality Technologies shipped a new platform version adding federated workload support for organizations that need to analyze sensitive data across separate legal entities. Intel and Duality also expanded the OpenFHE library in 2025 with new optimizations aimed at making the technology more accessible for privacy-preserving machine learning generally. Together with Google's HEIR release, these moves paint a picture of an industry moving from isolated academic prototypes toward a genuinely competitive commercial landscape, though one still bounded by real performance limits.

Pros and Cons of HEIR

Pros

  • Completely free and open source, with no licensing fees for commercial use of the compiler itself
  • Python frontend lowers the barrier for developers without a cryptography background
  • Built on MLIR, giving it a path toward broad interoperability across encryption schemes and hardware
  • Backed by real hardware accelerator partnerships, not just software benchmarks
  • Four working, source-available demo applications instead of only theoretical documentation
  • Active academic adoption already producing peer-reviewed research

Cons

  • Performance overhead remains extremely high compared to unencrypted computation
  • Public benchmarks so far are CPU-only; real hardware-accelerated numbers are not yet published
  • No demonstrated support yet for large-scale transformer or LLM inference
  • Still requires meaningful engineering effort to integrate into an existing production pipeline
  • Ecosystem and community support are smaller than mature libraries like Microsoft SEAL

What's Next for Encrypted AI

Google's blog post ends with a fairly modest promise: continued work to make homomorphic encryption easier to develop, faster to run, and more common across the industry. Realistically, expect the next twelve months to bring published latency benchmarks from Google's hardware accelerator partners, a slow expansion of the model types HEIR can compile, and continued academic output building on the framework.

The bigger question is whether encrypted AI inference becomes a background expectation the way HTTPS did for web traffic, invisible to most users but assumed by default in any serious product, or whether it stays a specialized tool reserved for the highest-stakes healthcare and financial workloads. Given how quickly the underlying cost curve has moved over the past few years, and given that three of the biggest cloud providers now all have a horse in this race, the former outcome looks increasingly plausible, even if it's still likely years away for anything running at consumer scale.

Frequently Asked Questions

What is Google HEIR?

HEIR stands for Homomorphic Encryption Intermediate Representation. It's an open-source compiler toolchain from Google that converts pre-trained AI models so they can run on encrypted data instead of plain text, without ever decrypting that data during processing.

Is HEIR free to use?

Yes. HEIR is open source and available on GitHub under Google's public repository, alongside the heir_py Python package that developers can install to start compiling models.

What is homomorphic encryption in simple terms?

Homomorphic encryption is a cryptographic method that lets a computer perform calculations directly on encrypted data and produce an encrypted result, without ever seeing the original, unencrypted information.

Is homomorphic encryption slow?

Yes, it carries significant overhead. Depending on the operation and model, homomorphic computation can run anywhere from roughly 100 times to over 1,000 times slower than the same operation on unencrypted data, which is why hardware acceleration and careful model design matter so much.

Who is HEIR for?

HEIR targets application developers who want to add encrypted AI inference without hiring cryptography specialists, hardware teams building homomorphic encryption accelerators, and academic researchers benchmarking new optimization techniques.

How is HEIR different from Microsoft SEAL or Zama Concrete ML?

SEAL and Concrete ML are libraries built around specific encryption schemes. HEIR is a compiler framework built on MLIR that aims to sit above multiple schemes and hardware backends at once, so it functions more like an interoperability layer than a single-purpose library.

Can HEIR run large language models?

Not yet at production scale. Google's initial demos focus on smaller, well-defined models such as recommendation systems, fraud detectors, and hotword detection. Full encrypted LLM inference remains an active research area across the industry.

Does encrypted AI inference protect against data breaches?

It significantly reduces the risk. Because the server processing the data never has access to the decrypted version, a breach of that server would only expose ciphertext, which is useless without the corresponding private key.

When will HEIR be practical for everyday apps?

Google says cost is falling quickly and points to partnerships with hardware accelerator companies as the next step toward practical latency. Most analysts expect narrow, high-value use cases in finance and healthcare to arrive first, with broader consumer use following as accelerator hardware matures.

Conclusion

HEIR isn't going to make your apps faster, and it isn't going to show up as a feature you can toggle on this week. What it represents is something quieter and arguably more important: Google handing the rest of the industry a shared, open-source foundation for building AI that genuinely never sees the data it's working on. The overhead is real. The Hacker News skeptics have a point. But a free, Python-accessible compiler that turns an ordinary model into an encrypted one is a meaningfully lower bar than "hire a team of cryptographers," and that alone is likely to pull a lot more experimentation into this space over the next year.

Whether HEIR becomes the industry standard or just one strong option among several from Microsoft, Zama, and Duality, the direction is now unmistakable. Privacy-preserving AI is moving from an academic curiosity toward something companies are actively racing to ship.

Related Reading on Mustrend

Sources

Know a developer who'd want to try compiling their first encrypted model this weekend? Share this with them.