Cloud Services for LLMs and Generative AI in Ukraine

When choosing a GPU server for Large Language Models, it is important to assess not only the number and nominal performance of the accelerators, but also the characteristics of the models being used, along with the related technical details. This is especially important when designing a comprehensive LLM platform. In this case, the critical parameters are the amount of available high-speed memory and the bandwidth of the data exchange bus between accelerators, or interconnect.

For example, the weights of a large LLM model can take up hundreds of gigabytes even before user context is taken into account. If they do not fit into the memory of a single accelerator, the system will distribute computation across several GPUs. That is why a server for LLMs must support high-speed interconnects such as NVLink.

At the same time, most LLM deployments are rarely limited to a single server. Sooner or later, expansion will be required, and this should be planned for in advance. For example, distributed high-speed storage may be needed for rapid weight loading, along with monitoring systems and tools for inference orchestration. Building, developing and maintaining such a system in-house is difficult and very expensive. Yet all the required resources are now available in the cloud. Any major LLM provider offers a complete set of capabilities for deploying and developing large language models.

Training an LLM from scratch is time-consuming and expensive. Today, businesses almost never set themselves this kind of task. Instead, when they need to create a corporate equivalent of a neural network such as ChatGPT, they take a ready-made foundation model from the many dozens available on the market and give it access to a body of closed corporate knowledge, without changing the architecture of the network itself. This knowledge usually consists of documents, databases, internal policies and similar sources. Cloud hosting for Large Language Models in such projects must provide secure and reliable connection of the LLM to these internal sources without the risk of sensitive data leakage. Accordingly, hosting for LLMs must integrate seamlessly with the organisation’s IT security system. The integration of the organisation’s own knowledge is carried out using the RAG approach, or Retrieval-Augmented Generation, which dynamically inserts the relevant documents into the user’s query at the moment of request.

For companies in Ukraine, an important factor remains the deployment of AI platforms within Ukrainian jurisdiction, so cloud hosting for Large Language Models must take this into account. However, data localisation is no longer a problem, as all major Ukrainian operators offer LLM hosting in the cloud within the country. This ensures not only that data is hosted without being moved abroad, but also that network latency is kept to a minimum.