LibreChat as a central AI platform for your company Blog

Once AI moves beyond isolated experiments, three decisions come up sooner or later. Which model should handle a task? Which company data may it access? And how should it work with that data so a single chat turns into a repeatable way of working?

With proprietary AI services, these decisions are limited to what the provider, the plan and the integration model allow. Access to internal systems also follows the technical and contractual terms of the respective service.

LibreChat is open source and therefore adaptable. Instead of adopting a ready-made environment, you build your own: with the models you want to use, with connections to your business systems and with permissions that fit your organization. The result is the architecture you need, and from it a working environment rather than a chat window. How sovereign it ends up being depends on exactly these decisions.

Choosing models by task and sensitivity

What users notice first is a shared interface for different models and services instead of separate subscriptions and conversation histories. LibreChat ships with native connections to the major providers; so-called custom endpoints let you add further OpenAI-compatible services. Behind them can sit a hyperscaler with a selectable region, a router, a direct vendor endpoint or a locally hosted model. For first local installations, Ollama is a good starting point. What matters, however, is not the provider's name: where data is actually processed depends on the contract and the specific endpoint.

This technical variety does not have to reach the users in full. In the interface they only see the models approved for their area of work. Clear names and an explicit note on the processing location make the choice easier. But even this limited choice does not have to sit with the users. For automatic selection, LibreChat can address a router like LiteLLM as its central model endpoint. It inspects every request and forwards it to the appropriate route. Non-critical content can go to any approved model, sensitive content exclusively to a model in the EU or in your own environment. LiteLLM's Sensitive Data Routing works rule-based and detects defined patterns or keywords. A model-based assessment can be added as an extra stage. Which variant makes sense depends on the use case and the level of protection required. Which data goes where, however, must not be decided by the model itself. This boundary has to be enforced technically and remain controllable.

Making company knowledge available on purpose

Once it is clear which model processes a request, the second decision follows: which internal information may it access? General model knowledge alone is not enough for many business tasks. The value emerges when information from databases, ERP, CRM or other business systems is added. Instead of copying content or uploading entire data sets, the required information can be retrieved directly from its source.

For this, LibreChat uses, among other things, the Model Context Protocol, MCP for short. The open standard defines how AI applications access tools and data sources. Such a tool can, for example, offer a search in the inventory management system or a specialist database. It describes which task it performs and which inputs it needs. When someone asks a matching question, the model calls the tool with the corresponding criteria. A service inside the company network then queries the business system and returns the result. The entire data set never has to be transferred.

That result, however, also enters the model's context and follows the previously defined model route. Model choice and data access are therefore directly intertwined: the connection determines which information is returned and in what scope, the model route determines where it goes. Access rights and data minimization must therefore be enforced in the tool itself and must not depend solely on the model's decision.

In principle, any system that provides a suitable interface can be connected. For common applications there are ready-made MCP servers, whose origin, scope of permissions and handling of credentials should be reviewed before production use. In LibreChat, such connections can be configured centrally or set up by authorized users directly in the interface. They can then be released selectively for individual people, groups or roles. Where no ready-made connection exists for a company's own business system, a custom MCP server comes into play. This is where we at wonk.ai are happy to help with our experience.

Building your own MCP server

Ready-made MCP servers cover common systems. The greatest value, however, often comes from custom connections to a company's own business systems. For this we develop MCP servers tailored to the specific use case. If a REST interface already exists, a first prototype can usually be built with manageable effort.

An MCP server can provide different building blocks. Tools are functions the model calls itself, such as a search in a specialist database. Resources provide content that can be brought into a conversation. Prompts provide prepared instructions or conversation starters that users select deliberately. In practice, the tools usually carry the main load, and not every AI application supports all three types equally.

With custom connections, the work does not end with the technical link. A tool must be described so that the model selects it in the right situations, and constrained so that it only exposes the data and actions that are required. Write access in particular needs clear permissions, input validation and traceable logging.

The presentation of results is evolving as well. So far, the model often renders returned data as text, so wording and structure can differ with every request. MCP Apps instead enable programmed interfaces such as tables, forms or interactive analyses directly in the conversation. The underlying data stays dynamic, but its presentation follows a defined logic and can be adapted to the corporate design. The model still decides when to use the tool; the application determines how the result appears.

MCP Apps have been the first official extension to the Model Context Protocol since January 2026. The platform in use, however, must explicitly support this extension. In LibreChat version 0.8.7, which we reviewed, this is not yet the case.

From data access to work results

Not every tool has to be built from scratch. Alongside custom MCP servers, LibreChat ships with features for documents and web search. Together they initially solve the same problem: they supply the model with information that is not part of its training knowledge. Both features are integrated into the interface, but need to be configured first and create their own data flows. With documents, the original file, extracted text, embeddings and the passages sent to the model can reside in different places. With web search, your employees' search terms reach the respective search provider.

Retrieving information, however, is only the first step. Often the model is supposed to keep working with the data it found, for example analyze a spreadsheet, create a chart, clean up a data set or run a scenario calculation.

This is exactly what the Code Interpreter is for. LibreChat relies on an open-source service licensed under Apache 2.0 that can run on your own infrastructure. The model writes code, the service executes it in an isolated environment. Unlike a purpose-built MCP tool, nobody has to know in advance which specific analysis will be needed. MCP can provide controlled access to the data, the Code Interpreter handles the flexible processing.

Where the code runs and how the resulting files are handled remains an architectural decision. The service can run in isolation on your own infrastructure. If you use an external provider's execution instead, processing location and retention follow that provider's terms. Which path fits is determined by the sensitivity of the data being processed.

Agents and skills organize the work

At this point the technical options are in place: which model works, which data it may access and which tools are available. A repeatable way of working, however, only emerges when these options are brought together for a specific task.

This is what agents are for: they bundle a model, suitable instructions, tools and data access. They can be set up for specific tasks and released selectively for individuals or groups.

Skills extend an agent with reusable procedures. A controlling agent, for example, can use one skill for the monthly report and another for scenario analyses. In quality management, a skill can define how documents are reviewed, deviations are classified and results are documented.

The difference is easy to summarize: the agent sets the frame, the skill describes the procedure. Individual features thus become repeatable workflows that can be maintained centrally and provided consistently to everyone involved.

What operations demand

From the outside, LibreChat looks like a single application at first. In operation, however, the platform depends on many components: user management, database and file storage, model endpoints and routers, document processing, MCP servers and the Code Interpreter. Each of these components has to be updated, secured, monitored and scaled as usage grows. The more models, agents and connections are added, the larger the number of dependencies becomes.

Every agent therefore creates not just a new feature but an operational promise. Model, data route, tools, permissions and instructions must keep fitting together over time. Updates are an operational process of their own: new versions can change configurations, interfaces or the behavior of individual features. Before an update you therefore need backups, traceable configurations and tests of the most important workflows.

Many failures remain invisible at first. The interface is reachable even though a model has stopped responding, an access key has expired or an MCP server has gone down. Central monitoring, for example with Prometheus and Grafana, should therefore not only check the application itself. It must also cover model endpoints, connected services and code execution, make errors and latencies visible and actively alert on incidents.

Monitoring also includes cost transparency. Beyond usage-based costs for models, web search, embeddings, storage and code execution, there are ongoing expenses for infrastructure, monitoring and maintenance. Consumption and costs should therefore be traceable per service, model and use case. Budgets and alert thresholds surface unexpected consumption and misconfigurations before they get expensive.

Only in live operation does it become clear whether the architecture actually works flawlessly. Requests must reliably reach the right model, tools may only return the data they are approved for, and agents and skills must keep working as intended after updates. The real achievement is therefore not installing an interface but keeping these components working together permanently.

Installation is only the beginning. Anyone who treats such a platform as a small IT project that is finished at go-live underestimates its technical and organizational complexity. Sovereignty does not come from self-hosting alone, but from reliable operations.

This is exactly the work we take on for our clients: from architecture and model routing to connecting business systems and ongoing monitoring.

Are you planning such a platform or want to stabilize an existing one? Let's start with an architecture and data flow check of your most important use cases.