Privatized large-model edge deployment application development system RTS-LQ-RAG001

RTS-LQ-RAG001 is a set of privatized large model edge deployment application development system implemented by the current popular RAG (Retrieval Enhanced Generation) technology, which can easily load the internal privatized database and knowledge base of the enterprise, and call the built-in language large model through the dedicated edge computing device to generate the enterprise's exclusive privatized large model, so as to create internal generative AI application scenarios.

Computing AI performance of edge computing devices:

● 2048 NVIDIA Ampere GPU cores

● Integrate 64 Tensor cores

● Provide 275TOPS INT8 AI performance

Edge Computing Device Physical Characteristics:

● size: 195mm x 210mm x 75mm, easy to deploy

● 60W low power consumption, up to 8 years long life cycle, for enterprise-level application scenarios

● Fanless design, wide temperature operation: -25C—+55C

● Rugged aluminum alloy shell, 7 x 24 hours uninterrupted and stable operation on site

Support language model:

● Tsinghua chatGlm 1, 2, 3

● Qwen ( 7B、14B )

● Baichuan 1, 2, 7B, 13B 4/8bit quantization

● LLaMa 1、2

● Scholars. Pu language

Application Development Management Module:

● Authentication login

● Conversational interaction

● Concurrency management

● Content filtering

● Records management

● User management

● ASR+TTS

● AVATAR

System structure:

System hardware platform

275 TOPS INT8 computingperformance, 12 cores and 64-bit ARM CPU, 2048 CUDA cores, 64 Tensor cores; 64 GB RAM, 64 GB external storage; Ubuntu 20.04 operating system, powered on auto-start.

Optional resources: WiFi/Bluetooth, 4G/5G, navigation module (GPS/Beidou/GNSS), large-capacity 2TB industrial-grade solid-state storage.

Application Scenarios:

●Q&A assistant for enterprise product customer service; You can upload enterprise product documents, materials, user manuals, specifications, manuals, etc., and use the private large model as an intelligent customer service to provide external services.

●Enterprise operation and maintenance knowledge and experience inheritance assistant; You can upload maintenance documents, manuals, processes, operation and maintenance knowledge, operation and maintenance procedures, etc., and use the private large model as an intelligent trainer to help newcomers in operation and maintenance.

●Q&A assistant for internal rules and regulations; You can upload the attendance system, administrative system, business process, financial system, etc., and use the private model to improve the internal operation efficiency of the enterprise.

●Legal Aid Q&A Assistant; National laws and regulations, industry laws and regulations, and so forth, can be uploaded, and private large models can be used to provide special legal aid services.

●Virtual TCM; You can upload various theories of traditional Chinese medicine, clinical experience of traditional Chinese medicine in hospitals, etc., and use private large models to provide traditional Chinese medicine consultation services to the outside world.

● Virtual teachers, counselors; You can upload school curriculum, teacher handouts, question banks, school rules and regulations, etc., and use private large models to provide services to students in the school.

●Policy Consultation Assistant of Government Affairs Center; You can upload local government policies and national policies, etc., and use private large models to provide 24-hour government affairs consulting services.

●Hospital guide and consultation assistant; You can upload various hospital procedures, hospital maps, medical guidance, etc., and provide patients with medical consulting services through private models.