Program Integrity Alliance's AI Strategy: Future-Proofing with the Model Context Protocol

Custom AI Can Age Fast

With a great deal of Fear of Missing Out (FOMO), thousands of organizations around the world committed to developing custom in-house solutions using Generative AI (Gen AI).

Soooo many chatbots.

It's understandable, the power of Large Language Models (LLMs) to interpret unstructured data and respond conversationally was a huge step forward in chat technology and Natural Language Processing (NLP). Also, being so new, there just weren't solutions available for organizations to use, so many rolled up their sleeves and started developing in-house.

This raised issues that go hand-in-hand with custom-developed software. Even if the AI part works well — and it often doesn't — most of the work in any software application is the engineering that surrounds it. Authentication, deployment, security, to name but a few, areas which often tend to be missed or developed as an afterthought. In many projects, this can be more work than the actual AI part. Also, being custom, they tend to cost more to support and maintain.

The IT build vs. buy debate is well-known, and the correct choice depends on a wide range of factors such as vendor lock-in, the uniqueness of the task, organizational technical capabilities, and more. But with GenAI being such a new technology, and in the absence of solid offerings in the market until now, many organizations jumped into custom solutions where previously they may have been more cautious.

AI Giants and the Limits of In-House Solutions

Another looming concern is that a lot of in-house GenAI solutions are sometimes competing with AI companies like OpenAI, Google, and Anthropic, which have millions, if not billions, of dollars at their disposal for researching and deploying state-of-the-art products that organizations can use right away. This is especially true for general-purpose (i.e. "wide") applications of AI, such as knowledge retrieval chatbots.

For example, a year or two ago, if an organization wanted to carry out deep research on its own internal documents, there wasn't really anything mature in the market for that. It required custom development of LLM workflows using open-source tools (e.g., LangChain), user interfaces, backend engineering, and more. However, most major AI vendors now offer deep research capabilities, and the ability to connect to external data sources is being actively progressed along with enterprise solutions to connect to organizational data.

Putting it another way, many GenAI custom solutions that took a year or two to develop may well be superseded by off-the-shelf functionality offered by the major AI companies, especially when one considers the rapid pace of AI innovation. What will next year bring?

AI Still Needs AI-Friendly Data, Enter MCP

If we assume commercial AI platforms will become more versatile and support many of the use-cases being pursued in-house by organizations, and organizations like federal agencies pivot towards using these off-the-shelf tools, then the emphasis shifts from developing user interfaces and orchestrating LLMs to instead providing the data (and tools) for off-the-shelf AI solutions to use.

This has been advanced recently with the advent of the Model Context Protocol (MCP), a standard that lets AI systems securely connect to data, tools, and instructions for using that data through an MCP server. It might sound super technical, but at its core, it's just a way for users to customize what an LLM can do, making it easier to complete tasks like analyzing information or performing searches of tailored data sources. MCP allows this by providing a consistent way for the LLM to interact with an MCP server, which maintains details about what resources are available and how to use them. A common analogy is that MCP is like a USB port for Large Language Models to 'plug in' to external tools and data.

At PIA, we have been on the road towards using MCP from the very start. We make a wide range of open U.S. government data sources AI-friendly. That includes collecting, processing, standardizing, and augmenting data with AI-generated tags so that they are easier to analyze and use. Our aim is to support our communities in government, Congress, and civil society to gain quicker insights and make more evidence-based decisions.

That said, we are a little cautious about new protocols, as these have a habit of evaporating if they don't gain sufficient momentum. There are also teething issues around security to iron out. Invest in them too soon, and there is a risk of wasted work. However, in the last 6 months, we have seen MCP explode. It is now either supported or soon to be supported by competing vendors such as Anthropic, OpenAI, Perplexity, and many others, and is rapidly becoming an industry standard for AI, with big 3rd party vendors writing MCP servers.

Are we seeing an App Store Moment for AI?

Until recently, MCP servers have been processes you run on your local computer. As such, they were a bit fiddly to set up and not very friendly for non-technical users, as well as being less secure because they run 3rd party code on your computer. In the last couple of months, however, there has been movement by the big vendors to also integrate with remote MCP servers, connecting to tools and data on the web. This opens up the possibility of a marketplace of MCP servers, where the user can choose from a community hub, a bit like choosing apps in the app store. The AI platform — e.g. ChatGPT, Claude, Perplexity — can take care of the LLMs, complex workflows, and user experience, with the data and tools provided by external connections to MCP servers.

We aren't quite at the App Store stage yet, but the signals are strong. One of the most compelling is that Docker — who already manage a huge repository of infrastructure patterns in Docker hub — have launched in beta Docker MCP Catalog, which has had over 1 million MCP server downloads in just a few weeks.

Docker MCP Hub, a repository of MCP Servers, has some very powerful MCP servers that do more than just access data; many will perform tasks and interact with popular applications. Importantly, it also addresses some of the security concerns around running local MCP code, implementing review, security protocols, and certification of MCP servers.

Docker MCP Hub is still in beta, and Docker tends to be for the more technical user, but it illustrates the momentum for community resources like this. It's not inconceivable this will progress to be more user friendly to non-technical users, with an App Store a person can access to add skills for their AI to use.

PIA, an 'MCP First' Organization

As a nonprofit, it is critical that we build tools that will last and use our limited resources wisely, so the work we do needs to have the biggest impact and also sustain through continued uncertainty as the AI revolution charges on. What we want to avoid is building solutions that go out of date before they are launched, so focusing on first developing AI tools using MCP seems a sound choice.

MCP is certainly not perfect yet; the technology is evolving quickly, and there is still a lot of variability in how it's implemented on different platforms. However, given the observed momentum, our approach when creating a new AI solution is to first add the required skills to our MCP server, then use that for the final product. This means the MCP part can be reused on a wide range of AI platforms and even for different use cases.

Our first MCP server tool is the ability to search in-document across hundreds of thousands of Government public reports, articles and oversight recommendations which PIA has gathered, enriched with AI, and organized for fast search. This is just a first step, though. We are progressing a roadmap of specialized tools, such as providing more insight for AI, based on our recommendations spotlight.