Edge computing models light up new AI business formats

        Driven by deep learning, the third wave of artificial intelligence is on the rise, and the emergence of ChatGPT has added another fire to AI.and generative AI based on big modelsis becoming the windfall of the next golden decade of artificial intelligence，promoting another big change in human productivity.

        In order to meet such large-scale parameter calculations, it is often necessary to use advanced distributed training technology to distribute model training tasks to multiple GPUs or servers for parallel computing to improve the training speed and efficiency, which consumes a lot of money and costs. The latest breakthrough achievement of porting large models to Jetson Orin, an embedded AI platform known for its low cost and small size, is a breakthrough achievement, and the Feiyun Smart Box RTSS-X102, independently developed and designed by Realtimes , has pioneered the adaptation of large model LLM and Stable Diffussion. The application of large models in edge computing platforms will light up the new AI business format where everything is intelligent and accelerate the empowerment of thousands of industries by artificial intelligence.

        LLM models possess the ability to understand and generate human language, they can learn language patterns from large amounts of text data, and are able to generate new, rational text. This capability has led to applications in many fields, such as natural language processing, machine translation, dialogue systems, etc. In addition, the LLM model can also be used for code generation, which can help developers generate code quickly and improve development efficiency.

        In addition, LLM models can also be used for image generation, which can help people quickly generate realistic pictures or videos. For example, using DALL· Models such as E2 or Stable Diffusion can automatically generate images or videos that meet the requirements based on the user's text description.

        text-generation-webui is a web UI developed based on Gradio, which is mainly used to run almost all available LLMs (large language models). This tool supports LLMs in different formats, such as GGML or GPTQ. The operation effect of the RTSS-X102 is as follows:

        Stable Diffusion is a deep learning model for generating images, and it is an improved version of the Diffusion model. By introducing the hidden vector space, the problem of Diffusion speed bottleneck is solved. In addition to being used specifically for Wensheng mapping tasks, it can also be used for drawing diagrams, specific character portrayals, and even super-scoring or coloring tasks.

        In general, to run a model on Stable Diffusion, you need to make sure that you have python and pip installed, and then you need to install some dependency libraries, including numpy, PyTorch, torchvision, etc. The stable diffusion model can then be downloaded and imported, and trained and inferred. The jetson-container in the generative AI lab has done all the work, and all we need to do is pull the image, open the browser, and access the http://0.0.0.0:7860

        The image generation model Stable Diffusion generates the corresponding image output through text input, so that users can generate images that meet their needs through text descriptions. An example of the result/output is as follows:

        Feiyun Smart Box X102 is a visual processing platform with powerful environment perception designed and developed for specific target customers based on NVIDIA Jetson AGX Orin, which has powerful computing power, video engine and rich interface resources. The core processor delivers up to 275 INT8 TOPS of AI computing power from Ampere-based GPUs and DLA, bringing powerful performance and AI performance, delivering server-class performance at the edge. With exceptionally powerful performance and leading power efficiency, Jetson Orin modules can run the NVIDIA AI software stack and power the next generation of demanding edge AI applications.

        According to data released by NVIDIA, generative AI has demonstrated good performance at the edge, as shown in the figure below:

Figure 1.Inference performance of leading generative AI models on Jetson AGX Orin

        Feiyun Smart Box X102 is a leading edge computing device, and its biggest highlight is the groundbreaking adaptation of generative AI models. With a highly optimized hardware and software architecture, the X102 is able to efficiently process massive amounts of data and achieve complex tasks. This enables a wide range of AI applications to be developed and deployed at the edge with extreme speed and efficiency. In the future, we will test and run some other models, such as MiniGPT4, EfficientViT, NanoSAM, Audiocraft, etc., looking forward to your attention and communication;